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1.  INTRODUCTION 


1. 1.  Motivating  Problem  -  Unit  Commitment  in  Power  Systems 

Meeting  the  load  on  a  large  scale  power  system  network  involves 
the  scheduling  and  control  of  multiple  generating  units  with  the  primary  goal 
of  minimizing  the  total  cost  of  generation.  Many  constraints  and  factors 
must  be  taken  into  account  and  problems  of  scheduling  involve  time  scales 
from  several  years  for  future  construction  planning,  to  a  year  for  maintenance 
scheduling,  to  daily  or  weekly  scheduling  of  the  units  wnich  are  available 
for  meeting  the  current  demand.  A  totally  integrated  scheduling  methodology 
taking  into  account  all  such  factors  involves  computational  requirements  far 
beyond  the  practical  limitations  of  computing  technology.  Consequently,  each 
aspect  of  the  problem  has  been  previously  investigated  with  the  relationship 
to  other  time  scales  being  only  heuristicly  and  intuitively  involved. 

One  of  these  problems  has  become  known  as  the  unit  commitment 
problem.  This  deals  with  the  scheduling  on  a  daily  or  weekly  basis  of  the 
generating  units  which  are  to  supply  the  power  to  meet  the  predicted  demand. 
Since  various  units  operate  at  different  efficiencies,  the  cost  of  meeting 
the  demand  can  be  affected  by  the  selection  of  the  generating  units  to  be  on 
line  at  any  given  time.  Obviously,  the  availability  of  units  is  affected  by 
maintenance  scheduling  and  forced  outages.  Assuming  such  schedules  are  known, 
the  commitment  problem  Involves  the  selection  of  the  optimal  generation 
schedule  from  an  enormous  number  of  possibilities.  This  problem  itself  is 
much  too  large  to  be  solved  completely  for  a  practical  system.  Therefore, 
suboptimal  schemes  have  been  suggested  and  implemented  for  current  applications 


Currently,  unit  commitment  is  done  using  interactive  computer 
programs  which  allow  the  system  operator  to  employ  the  computer  to  enhance 
his  own  intuitive  knowledge  and  experience  concerning  reasonable  generation 
schedules  [7].  A  relatively  small  amount  of  theoretical  investigation  has 
been  reported  concerning  many  important  aspects  of  the  problem.  The  suboptimal 
schemes  which  have  been  researched  and  implemented  certainly  use  sophisticated 
optimization  theory  whenever  possible,  including  integer  programming  [2,5,9,10] 
and  dynamic  programming  [4] .  However,  the  approach  normally  taken  has  been  to 
cast  the  unit  commitment  problem  in  a  form  which  allows  the  application  of 
existing  optimization  theory.  Only  recently  has  the  particular  problem  of 
unit  commitment  been  used  to  motivate  research  which  seeks  new  ideas  to  include 
important  aspects  of  the  problem's  unique  nature. 

In  two  papers,  Turgeon  [11,12],  suggests  that  the  shut-down  and 
start-up  of  thermal  units  may  be  modelled  dynamically  and  that  the  control  of 
the  off-line  units  may  be  considered  as  individual  optimization  problems.  He 
suggests  that  the  systems  which  are  on-line  operate  at  a  given  state  which 
determines  the  end  conditions  for  the  control  problem  defined  for  the  off-line 
systems.  In  the  following,  these  concepts  are  incorporated  into  the  generic 
model  for  a  general  scheduling  and  coordination  problem  for  multiple  dynamic 

systems.  The  issues  addressed  are  motivated  by  the  unit  commitment  problem, 
but  are  of  a  strictly  theoretical  nature.  The  motivation  for  such  an  investi¬ 
gation  is  two-fold. 

Firstly,  the  general  problem  formulation  demands  a  fundamental 
investigation  involving  basic  properties  which  are  assumed  or  overlooked  in 
more  pragmatic  approaches  to  the  problem.  Often  these  properties  are 


intuitively  obvious  and  hence  not  of  necessary  consequence  when  the  goal  of 
research  is  to  produce  a  working,  practical  suboptimal  scheduling  scheme. 

As  is  born  out  in  the  following,  such  properties  often  require  considerable 
care  and  effort  to  state  and  demonstrate  rigorously.  Secondly,  the  results 
presented  are  directed  towards  implementation  as  another  step  in  the  develop¬ 
ment  of  practical  computational  schemes  to  solve  the  unit  commitment  problem. 
Since  all  current  methods  produce  suboptimal  solutions  and  the  global 
optimization  problem  remains  impractically  large,  the  results  presented  here 
enable  one  to  take  any  given  schedule  and  to  improve  upon  it  through  a  type 
of  gradient  search.  The  theoretical  analysis  of  such  an  approach  is  interesting 
and  useful  in  its  own  right. 

In  the  remainder  of  section  1,  the  generic  optimization  problem  to 
be  considered  throughout  the  report  is  first  described  in  words  and  then  for¬ 
mulated  mathematically.  It  will  be  clear  that  the  unit  commitment  problem 
for  power  systems  discussed  above  directly  motivates  and  can  be  formulated 
within  the  general  model  given.  However,  since  the  primary  interest  of  the 
research  was  of  a  theoretical  nature  the  unit  commitment  problem  and  the 
application  of  the  results  presented  to  that  problem  will  not  be  mentioned 
until  the  concluding  section  of  the  report.  In  that  section,  the  application 
of  the  theoretical  work  and  some  practical  aspects  of  the  thermal  unit  modeling 
will  be  discussed. 

1.2.  Generic  Problem  Description 

Large  scale  systems  often  consist  of  several  subsystems  which  must 
be  operated  in  a  coordinated  fashion  so  that  the  aggregate  performance  of 
the  subsystems  meets  a  given  objective.  The  particular  structure  of  the 


large  scale  systems  to  be  considered  in  the  following  involves  several 
essentially  autonomous  dynamic  subsystems  for  which  the  local  controls,  are. 
selected  in  response  to  a  policy  dictated  by  an  overall  coordinator.  Each 
subsystem  may  produce  an  output  over  a  certain  range  of  output  levels  while 
maintaining  a  particular  operating  steady-state.  The  coordinator  selects  the 
subsystem  output  levels  so  that  the  sum  of  the  outputs  from  the  participating 
subsystems  equals  a  given  demand  to  be  met  by  the  large  scale  system  (Fig.  1). 

Since  the  subsystems  operate  at  a.  steady-state  while  meeting  the 
output  level  dictated  by  the  coordinator,  the  coordination  of  the  participating 
systems  is  determined  so  as  to  minimize  the  total  steady-state  operating  costs 
of  all  subsystems  involved.  Hence,  even  if  the  overall  demand  varies  with 
time,  the  optimal  output  level  for  each  participating  unit  may  be  selected  by 
solving  a  static  optimization  problem  for  each  point  in  time. 

The  coordinator  is  allowed  at  each  point  in  time  the  option  of  not 
including  some  of  the  subsystems  at  all  in  the  steady-state  output  coordination 
If  for  an  interval  of  time  a  particular  subsystem  is  released  from  participat¬ 
ing  in  the  aggregate  output  of  the  overall  system,  it  becomes  autonomous  and 
need  not  maintain  the  operating  steady-state  until  the  coordinator  once  again 
demands  an  output  from  it.  Hence,  the  local  control  is  selected  so  that  the 
subsystem  is  back  in  the  operating  steady-state  when  needed.  It  is  assumed 
there  is  cost  incurred  in  operating  the  subsystem  during  such  an  "off-line" 
interval  of  time  and  the  local  control  is  determined  so  as  to  minimize  this 
cost. 

The  coordination  policy  involves  two  levels,  the  second  being 
determined  in  light  of  the  first.  The  coordinator  must  select  the  subsystems 


which  will  participate  in  the  output  production  for  each  point  in  time. 

Then,  each  local  control  is  determined  to  either  produce  an  output  which 
minimizes  the  overall  steady-state  production  cost  or,  during  the  intervals 
of  time  when  the  subsystem  is  off-line,  the  local  control  is  determined  by  a 
dynamic  optimization.  The  coordinator  must  select  the  participation  policy 
so  as  to  minimize  the  total  cost  incurred  by  all  subsystems  during  both  the 
off-line  and  production  or  "on-line"  time  intervals. 

1.3.  Mathematical  Model 

Suppose  the  system  is  to  operate  during  the  time  interval  [0,T] 
and  the  demand  to  be  met  by  the  sum  of  the  subsystem  outputs  is  given  by  the 
continuous  function  D:[0,T]  -R1.  The  dynamics  of  the  N  subsystems  are 
described  by  the  equations 


x1  -  fi(xi,ui,dt  .  vt) 


i*l, . . . ,N 


where  x*"  and  u  are  vector  functions  of  time  representing  the  state  of  the 

i  ni 

ith  subsystem  and  its  local  control,  respectively,  with  x  (t)  fcR  and 
i  “i 

u  (t)  £R  .  The  scalar  function  d. (t)  is  the  output  demanded  from  the  ith 


subsystem  when  it  is  on-line  and  the  coordinator  must  select  d(t) 
to  satisfy  the  constraints 


dL(t) 

Vt) 


d^d^tJSd^  i-l,...,N,  t  fc  [0,T] .  (2) 

The  scalar  function  v^  :  [0,T]  -»  £0 , l]  is  selected  by  the  coordinator  to  indicate 
when  the  ith  subsystem  is  on-line  or  off-line  according  to 


vt(C) 


0  -•  ith  subsystem  off-line 


1  -•  ith  subsystem  on-line. 


******  ' 


Hence,  when  the  ith  subsystem  is  off-line  the  output  demand  is  d^(t) *v. (t)  *  0. 

Each  subsystem  can  meet  the  demand  d. (t)  when  it  is  on-line  and  .in 

-i  ni 

an  operating  steady-state  denoted  by  x  €r  ,  i*l,...,N.  The  control  at  each 

i  mi 

point  in  time  must  be  in  a  given  constraint  set  U  CR  ,  i*l,...,N.  Hence, 
for  all  d^  satisfying  (2)  there  is  a  u*  £u*\such  that 

0  =  f^.u^dj)  (4) 

which,  because  of  (1),  implies  steady-state  operation.  Although  there  may 

be  many  admissable  u*-  which  satisfy  (4)  for  a  given  d^,  it  is  assumed  that 

for  each  demand  d^  there  is  a  predete mined,  unique  u^  which  is  selected  as 

l 

the  local  control  to  satisfy  (4).  I.e.,  there  is  a  function  Y*  :  R1  ~*R 

which  determines  the  on-line  control  for  the  ith  subsystem  given  the  demand, 
d Therefore, 

fi(^t,Yi(di),di) -0  1  d1sdt*di.  (5) 

Let  the  cost  per  unit  time  of  operating  the  ith  subsystem  with  control 

i  i  ai+mi  1 

u  while  in  state  x  be  given  by  the  scalar  valued  function,  :  R  -R  . 

—4  i 

Then  for  steady-state  operation  the  cost  per  unit  time  is  given  by  L^(x  ,u  ), 
or  the  cost  can  be  written  directly  as  a  function  of  the  demand  d^,  cp  (d^), 
given  by 

W  "  Li(^i^i(di)).  (6) 

Now,  for  a  given  time  t*=[0,T],  suppose  the  coordinator  has 
determined  the  subsystems  which  are  to  be  on  line  by  selecting  the  appropriate 
v(t)  t  tO,  1}N  where  v(t)  ■  [vi(t) ,...  ,vN(t)]  and  the  v^t)  are  defined  by  (3). 


8 


Let  £*[M]  denote  the  power  set  of  an  arbitrary  set  M,  i.e.  ^[M]  is  the 
collection  of  all  subsets  of  M.  Define  J  \  [o,l}^  as 

^(v)  -  {i  I  vt  -  1}  where  v*  [v^...,^] '  6  i.0,lJN.  (7) 

For  a  given  demand  D(t)  =  D,  the  vector  v(t)  =v  will  be  said  to  be  feasible 
for  D  if 

2  d  £  D  s  E  d . .  (8) 

i^(v)  i&(v)  1 

Clearly  v(t)  must  be  feasible  for  D(t)  for  all  t€[0,T]  and  this  is  a  con¬ 
straint  imposed  upon  the  selection  of  v(t). 

Given  a  feasible  v(t)  for  D(t)  the  demands  for  the  on-line  units 
are  chosen  to  minimize  the  sum  of  the  costs  for  those  units.  Let  C  :  RN+1'-*R*‘ 
be  the  minimal  cost  per  unit  time  for  meeting  D(t)  with  the  subsystems 
^[v(t)].  I.e. 

C[D(t),v(t)]  -  min  _  E  cp.  (d  )  (9) 

41idiSdi  i6/[v(t)] 

subject  to  the  constraint 


2  d.  -  D(t) 
i^[v(t)] 


(10) 


Consider  now  the  problem  faced  by  the  controller  of  the  ith  sub¬ 
system  when  v^t)  -  0  on  an  interval  t  and  v^t)  switches  from  1  and 

to  1  at  times  tj^  and  1 2>  respectively.  The  control  u1  on  the  interval 
(t^yt^)  must  minimize  the  cost 


9 


Ji(t2'tl’uZ)  -/  ZL1(xi,ui)dt 


(U) 


subject  to  the  two  point  boundary  value  (tpbv)  problem 


with 


^-fV.uSo) 


x^(t^)  ■xi  and  x^Ct^)  *x*\ 


i  * 

Let  the  minimal  value  of  J^(t2_ti»u  )  be  denoted  by  ^“t-^)  if  it  exists. 


I.e. 


★  i 

(C2"C1^  *  min  Ji(t2*ti»u  ). 
ui(t)6Ui 


(12) 


Since  all  constraints  and  the  cost  per  unit  time,  L^x^uS  are  independent  of 
time,  it  suffices  to  consider  J  (•)  as  a  function  of  the  time  interval  length. 


w 

To  write  the  total  cost  of  selecting  a  particular  policy  v(t)  for 

i  Ni 

t([0,T),  assuming  v(t)  is  feasible  for  each  D(t),  let  [t  }  ,  be  the  times 

i 

for  which  v^(t)  switches  from  0  to  l,  and  let  be  the  times  vi(t) 

switches  from  1  to  0.  Assume 


«i 


112  2 


(13) 


which  is  always  possible  for  any  v^(t)  (since,  if  it  is  not  true,  (13)  may 
be  made  to  hold  for  another  v(t)  for  which  v^(t)  -Vj,(t)  almost  everywhere 
(a.e.)).  Then  the  total  cost  of  the  off-line  operation  of  the  ith  subsystem, 

^i  £  vi  (t)  J ,  which  is  actually  a  functional  of  v^(t)for  0st*T,  is  given  by 

N, 


'iIvi(t)]  “k-1 


(14) 


m  1^  itM  na ItiMri  «i  .  ■ . 


10 


So,  letting  C>[v(t)]  denote  the  total  cost  £or  both  the  off-line  and  on-line 
systems  over  [0,T]  for  policy  v(t),  we  have 


T  N 

4[v(t)]  -J  CtD<t),v(t)]dt+t£^ttvt(t)]. 


(15) 


o 

The  coordinator  wishes  to  select  v(t)  to  minimize  &[v(t)]  of  (15). 
Note  that  the  definitions  of  C[D(t),v(t)]  and  ^[v^(t)J  actually  involve  the 
costs  incurred  by  the  subsystems  and  the  local  control  policies,  u*(t).  How¬ 
ever,  once  v(t)  is  selected,  u*(t)  is  determined  directly  from  the  steady- 
state  and  off-line  optimization  problems  defined  above.  Hence,  tne  coordinator 
selects  the  policy  v(t)  to  minimize  ^[v(t)]  subject  to  the  subsequent  optimiza¬ 
tion  problems  which  determine  the  local  controls,  u*(t).  This  is  a  leader- 
follower  differential  game  which  happens  to  be  of  a  special  sort  defined  and 
discussed  in  the  following  section. 


2.  RELATIONSHIPS  BETWEEN  LEADER-FOLLOWER  AND  TEAM  GAMES 
2.1.  General  Results 

It  is  well-known  from  the  theory  of  leader-follower  games  that,  in 
general,  the  minimal  cost  obtainable  by  the  leader  is  not  necessarily  the 
absolute  minimum  cost  which  would  be  obtainable  were  the  leader  and  followers 
to  cooperate  in  selecting  policies  with  the  sole  objective  of  minimizing  the 
leader's  cost.  This  latter  situation  where  all  participants  in  a  game  cooperate 
to  minimize  a  single  cost  will  be  referred  to  as  an  optimal  team  policy.  The 
following  discussion  leads  to  a  general  condition  for  which  an  optimal  leader- 
follower  policy  actually  gives  a  solution  which  is  equivalent  to  an  optimal 
team  policy  with  respect  to  the  leader's  cost. 


Consider  Che  following  Cwo  problem  definitions,  Pr  1  and  Pr  2 


Pr  1.  A  game  with  N+l  participants  is  defined  as  follows.  One  player,  the 
leader,  selects  a  policy,  v,  from  a  decision  space  V  and  the  remaining  N 
players,  the  followers  select  their  policies,  u*“,  from  decision  spaces  U^(v), 
i-l,...,N.  As  indicated  by  the  notation,  the  decision  space  for  the  ith 


follower  depends  upon  the  particular  policy  selected  by  the  leader,  but  is 
independent  of  the  policies  of  the  other  N-l  followers.  Each  follower  chooses 


its  policy  once  the  leader's  policy  is  declared  to  minimize,  if  possible,  the 

cost  J^v.u1)  which,  as  indicated,  is  a  function  of  the  leader's  policy  and 

the  particular  follower's  policy,  but  is  independent  of  the  other  N-l  followers 

* 

policies.  Define  the  optimal  followers'  costs  given  v,  J^(v),  as 


J*(v) 


inf  J^v.u  )  i  *  1, . . .  ,N. 


u i€ui(v) 


It  is  assumed  the  infinum  exists  in  (16)  and  subsequent  definitions  in  the 
sequel. 

Define  :  v"*<?{u^(v)}  for  i-1,  ...,N  as 

T^v)  -  UWOO  I  J^v.u1)  -  J*(v)}.  (17) 

Note  that  it  is  possible  for  T^v)-?  for  some  i6ll,...,N}  and  v  €v. 

The  leader  wishes  to  minimize,  if  possible,  a  cost,  J0(v,u),  where 
u  «  [u\ . . .  ,uN]  fcU(v)  and  U(v)  »U^(v)X  . .  .X  UN(v) .  Define  the  set  VcU  as 

V  -  (v  £v  i  Tt(v)  * 0  V  i«l,...,N3.  (18) 

The  leader  will  only  consider  policies  in  V,  i.e.  policies  for  which  all 
followers  can  achieve  their  optimal  costs.  If  the  problem  will  be 

considered  ill-posed  and  such  cases  will  not  be  included  in  the  subsequent 


2 


discussion. 

The  optimal  value  of  J0(v,u)  for  Che  leader-follower  game  will  be 
denoted  by  J^_j,  and  is  defined  as 

JL-P  “  J  (v,u)  (19) 

W  ° 

u  6l(v) 

where  T  :  V -ti>lul(v)}  x^tu2(v)}x...x^iuN(v)}  is  defined  by 

T(v)  »T1(v)XI2(v)X...X^(v).  (20) 

A  sec  of  policies,  (v,u),  for  Che  N+l  players  will  be  called  a 
solution  to  Pr  1  if  v  6V,  u6I(v),  and  J  (v,u)  «J*  .  The  solution  set  of 

'**  O  ""  L"l! 

Pr  1,  E^,  is  then  given  by 

ri  *  Uv,u)  £V  xU  (v)  IvfcV,  u£T(v),  JQ(v,u)  (21) 

Pr  2.  A  game  with  N+l  participants  is  defined  with  policies  v  and  u  and 

policy  spaces  V  and  U(v)  as  defined  for  Pr  1.  An  optimal  team  policy  is 

sought  to  minimize  Jq(v,u)  defined  in  Pr  1.  The  optimal  value  for  this  team 

* 

game  is  denoted  by  J^,  and  is  given  by 

J*  -  inf  J  (v,u).  (22) 

T  vfcV  0 

u  Su(v) 

A  set  of  policies,  (v,u),  for  the  N+l  players  will  be  called  a 
solution  to  Pr  2  if  v€V,  u  €u(v)  and  JQ(v,u)»JT.  The  solution  set  of  Pr  2, 
^2>  Is  Chen  given  by 

^2  “  Uv,u)  €V  xU(v)  I  Jo(v,u)  •  J*}. 


(23) 
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The  following  two  lemmas  ere  Immediately  evident  from  the  above 
definition*. 

*  * 

Lemma  1.  For  JL-j.  and  defined  by  (19)  and  (22),  respectively. 


*  „  * 

JT  ‘  JL-F* 


(24) 


Proof.  (24)  follows  from  the  fact  that  the  inf  imam  in  (22)  is  taken  over  a 
larger  class  of  (v,u)  than  that  of  (19). 

_2.  If  then 


(25) 


where  and  are  defined  by  (21)  and  (23),  respectively. 

Proof.  If  (v,u)  feE.,  then  v(VCV  and  u£l(v)cu(v),  and  J  (v,u)  «J*  *J*, 

by  hypothesis,  therefore  (v,u)  fcZj,  proving  (25). 

Now,  consider  the  class  of  problems  of  the  type  Pr  1  for  which  the 
following  property  holds.  Define  the  mappings  Tj  :  V  1  (v) }  x  ...  k^(UN(v)}  , 

j«l,2  as 

C ^(v)  •  lu  ^U(v)  I  u^  €T^(v)  Vi-l,...,Mj  (26) 

and 

r,(v)  -  lu  6U(v)  I  J  (v,u)  -  inf  Jft(v,u)}.  (27) 

u6U(v)  ° 

Property  tt;  a  problem  of  the  type  Pr  1  will  be  said  to  have  property  tt  if 

ri(v)  *r2(v)  V  v€V.  (28) 

In  words,  property  n  says  that  for  every  policy,  v  fcV,  of  the  leader, 
a  set  u  €U(v)  of  followers'  policies  minimizes  their  Individual  costs  if  and 
only  if  the  leader's  minimal  possible  cost  for  that  v  is  also  achieved  by  u. 
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Theorem  1.  If  a  problem  of  type  Pr  1  has  property  tt  and  either 

2^0 


V  -  V, 


•k  it 

JT  "  JL-F 


Proof,  Suppose  (v,u)  €  >  then  by  (23)  and  (22) 

*  *»  _  ~ 

JT  -  J. <v,u)  -  inf  J  (v,u). 

u€U(v)  ° 

Therefore,  by  (27)  and  property  tt,  u  6r^ (v)  *T^(v)  which  Implies  v  €V  and 
u€T(v).  Therefore,  by  the  definition  (19)  of  J*_F, 

JL-F  S  “  JT* 

ic  it  <«  m 

But  J_£J  „  by  lemoia  1,  hence  (31)  holds.  Furthermore,  J  (v,u)  »JT  _, 
implying  (v,u)  € 2^.  Therefore,  but  by  (31)  and  lemma  2  Z^cZ^, 

which  establishes  (32). 

Now,  assume  ty«V.  Note  first  that  if  (31)  is  established  for 

*  * 

this  case,  then  lemma  2  implies  (32).  By  lemma  1,  J  £  J  .  Assume 

X  li*f 

<  J*.F*  By  the  definition,  (22)  of  J*3v€v  and  ufcU(v)*J*  <  JQ(0,a)<  J*_p. 

A 

Consider  r^(0).  Since  V  ■  V,  T^(v)  is  non-empty,  hence  3  u’€r^(v).  By 
property  w,  ti’Gr^v)  which  implies 


JqCv.u’)  <  Jo(v,u)  <  JL_F» 


Therefore 


A  ^  ^  A  £ 

But  u'ST(v)  and  v€v,  so  this  contradicts  the  definition  of  J 

*  L“  r 

(31)  and  the  theorem  is  proved. 

2.2.  Relationship  to  the  Problem  of  Section  1 

Theorem  1  establishes  conditions  for  which  the  leader-follower  game 
is  virtually  equivalent  to  a  team  game  with  respect  to  the  leader's  cost.  It 
is  easily  seen  that  when  the  leader's  cost  consists  of  the  sum  of  the  individual 
followers'  costs  and  the  followers'  problems  are  independent  of  one  another, 
as  is  the  case  for  the  problem  described  in  the  previous  section,  then  the  pro¬ 
blem  is  of  type  Pr  1  and  has  property  it.  Furthermore,  for  the  problem  models 
used  in  the  sequel,  it  is  easily  shown  that  hypothesis  (29)  of  theorem  1  as 
well  as  hypothesis  (30)  hold.  Hence,  for  the  problem  being  considered  in  this 
work,  the  leader  in  fact  obtains  the  minimal  possible  cost  even  though  the 
local  controls  are  selected  to  solve  peripheral  optimization  problems. 

Although  the  result  of  theorem  1  may  appear  to  be  intuitively  evident 
for  a  problem  with  the  structure  described  in  the  previous  section,  the  property 
it  is  of  a  very  general  nature  and  may  often  hold  for  leader- follower  problems 
where  the  relationship  between  the  optimal  leader-follower  policy  and  the 
optimal  team  policy  is  not  immediately  evident.  It  is  often  the  case  that  the 
hypothesis  for  theorem  1  may  be  easily  verified  by  applying  known  results  in 
control  and  optimization  theory  to  the  given  problem  structure. 

3.  ANALYSIS  OF  THE  COORDINATION  OF  THE  ON-LINE  SYSTEMS 
3.1.  Discussion  and  Primary  Results 

In  this  section  the  properties  of  the  cost  incurred  by  meeting  the 
demand  through  steady-state  optimal  coordination  of  the  on-line  systems  is 
considered.  For  a  given  feasible  v  with  respect  to  a  demand  D,  the  minimal 
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steady-state  cost  of  meeting  the  demand,  C(D,v),  Is  defined  In  (9)  with  the 
constraint  (10).  Since  the  optimization  is  for  steady-state  operation,  the- 
integral  of  C[D(t),v(t)]  appearing  in  the  total  cost  function  of  (15)  is 
minimized  by  minimizing  the  integrand  for  each  t€[0,T]. 

The  properties  of  C[*,v]  for  a  constant  v  are  of  interest  for  analysis 
of  the  overall  optimization  problem.  Knowledge  of  the  nature  of  this  function 
allows  one  to  assess  what  type  of  approaches  should  be  used  to  solve  the  problem. 
In  the  following  the  major  properties  of  C[-,v]  are  given  in  a  lemma.  This  re¬ 
sult  is  discussed  as  it  relates  to  a  particular  approach  to  minimizing  the 
overall  cost.  Theorem  2,  proved  in  this  section,  can  be  used  to  generate  the 
gradient  of  the  cost  with  respect  to  the  switch  times  of  v(t) .  This  approach 
to  the  unit  commitment  problem  will  be  discussed  in  Section  5. 

The  remaining  primary  result,  theorem  3,  describes  the  properties  of 
the  function  C* ( - )  which  is  defined  below  as  the  minimum  cost  to  meet  a  demand 
D  with  any  feasible  v.  This  function  and  its  properties  are  of  interest  because 
it  represents  the  optimization  problem  when  the  cost  of  operating  the  off-line 
systems  is  disregarded.  Hence  it  provides  a  lower  bound  on  the  achievable  cost. 
Furthermore,  its  properties  are  useful  in  guiding  the  search  for  reasonable 
solutions  which  employ  integer  or  dynamic  programming,  as  will  be  discussed  in 
Section  5. 

Following  the  presentation  of  the  major  results,  the  full  proofs  and 
intermediate  results  are  presented  in  Section  3.2. 

Lemma.  (Lemma  6  in  Section  3.2).  If  <^(t),  i*l,...,N  of  (6)  are  monotone 
increasing  and  continuous  for  each  i,  then  C(*,v)  is  monotone  increasing  and 
continuous  on  any  interval  for  which  v€{0,l}  is  feasible. 

The  interval  of  feasibility  for  v  is  discussed  in  Section  3.2. 
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Knowledge  of  Che  properties  of  Che  optimal  cost  for  meeting  the  demand  as  a 
function  of  the  demand  will  be  useful  in  determining  efficient  algorithms  for 
minimizing  the  overall  cost.  The  following  result  is  a  consequence  of  the  con¬ 
tinuity  of  C(-,v). 

Suppose  for  D(t) ,  t€[0,T],  with  range  [I),D]  (the  range  is  defined  in 

Section  3.2),  there  exists  v^,  v2€{0,l}N  such  that  both  and  are  feasible 

N 

for  the  entire  range  of  demand.  Define  v(t,t  )€{0,1}  for  a  given  t  €(0,T)  and 

—  s  s 


every  t€[0,T]  as 


v(t,ts) 


0  <  t  <  t 


v.  t  <  t  <  T 
— I  s 


Then  v(t,tg)  is  feasible  for  D(t)  for  all  t£[0,T]  and  the  cost  of  the  steady- 

state  coordination  as  a  function  of  t  ,  J(t  ),  is  given  by 

s  s 

J(ts)  -  /*C[D(t),v(t,tg)]dt. 

A 

The  derivative  of  J(tg)  exists  and  is  given  by  the  following. 

Theorem  2.  For  J(t  )  and  v(t,t_)  defined  above,  for  any  t€(0,T), 

~  S  '  s 


3J(t  ) 

- £-  | 

3t  >t  -t 


CtDU),^]  -  C[D(t),v2]. 


Proof.  For  any  t€(0,T),  and  0  <  A  <  T-t, 


J (t+A)  -  J(t)  -  /{C(D(t) ,v(t,t+A) ]  -  C[D(t) ,v(t ,t) ] }dt 
o  —  — 

•  /t+^{C(D(t)  ,v^]  -  C(D(t)  ,v2]  }dt  -  (CCDU),^]  -  C[D(t)  ,v2]}*A 

for  some  t£[t,t+A],  where  the  last  line  follows  from  the  continuity  of  C(*,v^), 
i-1,2,  and  the  mean  value  theorem.  Dividing  by  A  and  taking  the  limit  as 
A  -*■  0  gives  (*)  for  the  right  derivative.  The  left  derivative  is  proven  to  be 


equivalent  to  (*)  in  a  similar  manner.  Q.E.D. 


A  generalization  of  Theroem  2,  combined  with  some  results  of  Section 
4,  can  be  used  to  define  the  gradient  of  the  total  cost  (15)  with  respect  to 
the  switch  times  of  an  arbitrary  v(t).  This  approach  to  the  optimization 
problem  is  discussed  in  Section  5. 

The  other  major  result  proved  in  Section  3.2  is  given  by  the  following 

theorem. 

Theorem  3.  If  C*(*)  is  discontinuous  at  D£[D,D]  then  the  discontinuity  is  one 
of  the  following  two  types : 

Type  1.  with 

D  -  t|s^di,  C*(D)  »  C(D, vL)  and 

C*(D")  >  C*(D). 

Type  2.  2  S2  fe^with  v2  *<y'”1(S2)3 

5  -  ,  C* (D)  -  C (D ,  v2 )  and 

C*^)  >  C*(D). 

Furthermore,  C  (•)  is  monotone  increasing  on  any  interval  in  [D,D]  which 
contains  no  points  of  discontinuity  of  C  (•)• 

'/C 

See  Figure  2  illustrating  the  two  types  of  discontinuity  for  C  (•)• 

•ft 

In  words,  theorem  3  states  that  at  points  of  discontinuity  C  (•)  assume  the 
lower  value  of  the  right  and  left  limits.  Definitions  of  certain  notation  in 
Theorem  3  are  given  in  Section  3.2. 

3.2.  Proofs  of  Primary  Results 

Before  considering  the  properties  of  C[D,v],  it  is  necessary  to 

N 

establish  conditions  for  which  there  is  at  least  one  feasible  v£i.0,lj  for 
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every  value  of  D(t),  tfc[0,T].  Let  D=  min  D(t)  and  D  *  max  D(t),  both 

tt[0,T]  t€[0,T] 

of  which  exist  since  D(t)  is  continuous.  The  range  of  demand  is  defined 
as  the  closed  interval  [D,D]. 

Definition.  A  set  of  N  subsystems  is  said  to  have  sufficient  capacity  with 
respect  to  (WRT)  the  range  of  demand  [D,D]  if  VD6[D,D]  SSfcv1^,  where 

*H-*[ll... 3 

2  d  6  D  s  2d  (33) 

its  i€S 


An  equivalent  statement  for  the  above  definition  would  be  to  say 
there  is  a  feasible  vt{0, 1}N  V  D£[D,D],  which  follows  from  (8).  Both  the 
definition  of  a  feasible  v  and  the  definition  above  of  sufficient  capacity  are 
based  upon  the  simple  fact  stated  in  the  following  lemma. 

Lemma  3.  Given  a  set  of  subsystems,  S  and  a  demand  D,  ad^  d^sd^sd^ 

for  each  i£S  where  2  d  *»D  if  D  and  S  satisfy  (33). 

its  ^ 

Proof.  Trivial. 

Next  we  state  a  necessary  and  sufficient  condition  for  a  set  of  subsystems 

to  have  sufficient  capacity  with  respect  to  a  given  range  of  demand. 

Lemma  4.  A  set  of  N  subsystems  has  sufficient  capacity  WRT  the  range  of  demand 

[D,D],  if  Sq  €t?  4 
N 

2d  *DS  2d  (34) 

16Q  1  itQ  1 


and  VsifeV 


3S2€^Nwhich  may  depend  upon 


D  *  2  d  <  D 

itSl 


Eds  2  d  <  2d. 

itS2-i  icSl  i  itS2  i' 


(35) 
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Proof.  Necessity.  Clearly  (34)  holds  by  the  definition  of  sufficient  capacity. 
For  S^  as  in  (35)  define  as 

“'l  =  *-S  6t<N  ^  ils  -i  > 

If  a', =  0,  let  S„  €<?„  be  the  set  of  subsystems  which  satisfies  (33)  for  D=D, 

J-  L  N 

then,  since 

2  d.  i  2  d.  <  ds  2  d 
itS2_ i  itS^  i  itS2  i 

If  y,  #«>, since  it  is  finite  2  D,  *min{  2  d  }  and  D.  >,  2  d..  Let  D.  ®mln[D.,D} 
_  1  St-'i  1  L  ltbl  1  1  ' 

and  for  D-j  6  (i(^  d^.D^  let  S2  6^  be  the  set  for  which  (33)  holds  for  D=D^ 

and  S  ■  S2 .  By  lemma  3,  S2  giving 

2  d  s:  2  d  <D  s  2d. 
ies2%  ifcS^i  U3  its2ai‘ 

Sufficiency.  Assume  (34)  and  (35)  are  true  and  let  D  6 [D,D] .  If  D  =  D,  (33) 
is  satisfied  for  S  =*Q,  as  in  (34).  Suppose  D  £  (D,D]  and  define  the  set  «*(D)  as 


*(D)  -iS  <=<?N  I  .2,d.  <  DJ. 


If  Qiii(D),  (33)  is  satisfied  for  S=Q.  If  Q  ^(D)  ^  6ji(D)3 


Si’^sWd)  16Sdl)  and 


2‘iiA  “  <  D  ■=  5‘ 

Therefore,  by  (35)  2  S2  6^  3  L^s  ii  s  it?s  dj/  Hence,  S2  £ii(D)  and  (33) 

is  satisfied  for  S»S2.  Q.E.D. 

For  the  subsequent  work  we  make  the  following  assumption  concerning 
the  cost  functions  ^(d^),  i“l,...»N,  defined  by  (6).  For  each  ifc(l,...,Nj 
the  function  9^(0  Is  continuous  and  monotone  increasing  on  the 
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domain  [d^d^].  Certain  properties  of  the  optimal  cost  function,  C(D,v), 
are  presented  below  as  consequences  of  this  assumption  which  will  not  be 
repeated  in  each  lemma  and  theorem, 

Loimiia  5 ,  Suppose  v  is  feasible  for  D  and  For  any  j  fcS,  let 

S.  =  S  -  {j}  and  v.  Then,  if  C(*,v^)  is  continuous  on  the  domain  for 

which  v.  is  feasible, 

-J 


C(D,v)  *  min  [C(D-x,v  )  +cp  (x)]  (36) 

d‘SxSd!  3  J 

“j  J 

where 

dj  -  maxi.d,j,D  -  d^  (37a) 

and 

dj  -  rainUj,D  -  ^  d^  .  (37b) 

Proof.  From  (37a, b)  x  is  within  the  domain  of  cp ^  .  Furthermore,  is 

feasible  V  D-x  when  xfc[dj,dj]  since 


d-iS3i  4  x  s  D-i|s.ij  '■  >i«  A  SD-*‘  i4.df 


Both  C(’,v.)  and  9.(«)  are  continuous,  hence,  on  the  set  [d'd!]  the  sum  in 

J  J  J  J 

(36)  asi 
C(D,v), 


TC  "mm 

(36)  assumes  its  minimal  value  for  some  x  t[dj,d^].  From  the  definition  of 


C(D,v)  £  C (D-x  , \[j  )  +  9j(x  ). 

If  C(D,v)  is  strictly  less  than  C(D-x*,v^)  +  (x  ),  Sld^j t£g  5 ^  * dj^  * dt  V i  £ S, 
i€Sai*D’  and 

tfcsVV  <C(D-x*,vj)  +«Pj(**). 


■  ■i'HBi'in  i 


r 


11 


23 

Let  x  ■  d . ,  then  d,  £  x  x  d  and  D-x  ■  .  2  d  -*  2  d  £  D-x  £  E  a  and  hence 

J  J  J  ifc® j  *  lcbj  ” 1  1 

x€[d!,d!J.  Furthermore,  C(D-x,v.)  at*. X  so  we  have 
~ J  j  “J  1 

C  (D-5c, v  )  +  <Pj  (i)  <  C  (D-x*,  v  )  +  «pj  (x*) , 

* 

which  contradicts  the  definition  of  x  .  Therefore, 

C(D,v)  =*  C(D-x*,Vj)  +<Pj  (x*), 

and  (36)  is  proved. 

Lemma  6.  Forvfc{0,l}N  and  S*V(y),  C(«,v)  is  monotone  increasing  and  continuous 

on  lies^i’ils1!1* 

Proof.  Let  ||s||  denote  the  cardinality  of  the  set  S.  Then,  for  ||s||  *  1  the 

lemma  is  true  by  the  assumptions  concerning  the  9^(0 .  Suppose  the  lemma  is 

true  for  all  C(*,v)  when  iU(v)||*k  for  some  k  6  [  1, . . .  ,N-lJ ,  but  that  for 

Sk+1  where  ||sk+J|  * k+1,  C(*,yk+1)  is  not  monotone  increasing  on  [  2  dt, 

1  _  'x+l 

ttSk+*i]  Where  \+lm*  <Sk+l>‘  Then  3  D1*D2  €[i£SSk+14'  i6SEk+LZi]  *D1<D2 
and  C(D1,yk+l)  >C(D2,yk+1). 

For  any  j  6Sk+1,  define  Sk  »  Sk+1  -  {j}  and  v^V1^].  Then  ||sj|  *  k  »  1  and  by 
the  induction  hypothesis,  C (• ,yk)  is  monotone  increasing  and  continuous  on 

tifiA'iSA1*  Define> for  Aml>2 

K  K 

*1  -  Mxldj  .0,  -  and  dj  -  1nU1.D£  - 1 § 4^ , 

Lemma  5  implies  3  for  4-1,2,  x^[d^,d^]3  C(Dryk+1)  -  C(D£  -  xryk)  +  <p  (x^ 

-  min  _(tC(D^-x,yk)+9j(x)J.  Hence,  C^-Xj^) +9^  (x^  >C(D2  -  x^v^ 

-&  &  x  s  dx 

Consider  the  following  two  cases: 
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Case  1:  +  d±.  Then  DL  <D2  implies  ^  ^  *  di"x2  <D2"X2 

k  k 

and  hence  is  feasible  for  D^-Xj.  fiy  the  monotonicity  of  C(*,vk>» 

C^-x^v^  +9j  (x2)  «C(D2-x2>vk)  +cPj  (x2)  <C(Dj-xlf  vfc)  +  9j  (x^,  which  contradicts 
the  definition  of  x^. 

Dl<X2+i^i*  Let  5‘iS/i  8nd  X“Dl‘i6Sk^i*  ^  ^k 
is  feasible  for  D  and 


Furthermore, 


C(D,vk)  £  C(D2-x2,vk). 


hence,  x  is  in  the  domain  of  <?,(•)  which  implies  cp^  (x)  £  9j  (x2> .  Since  D-D^-x 


we  have 


C(Dl-x,vk)  +  9j  (x)  *C(D2-x2,vk)  +9j(x2)  <C(D1-x1,vk)  +  <Pj(x1), 
which  also  contradicts  the  definition  of  x^. 

Therefore,  C(*,vk+^)  is  monotone  increasing.  Right  continuity  of  C(»,vk+^)  is 

proved  as  follows.  Let  D  fc [ .  £  d,»<  2  d, )  and  define  d' »max[d,,D  -  .  £  d  j 

ltSk+l  lt:i,k+l  _  “  J  ifcSk  i 

and  d'  *min{dj  ,D  -  d^}  .  gy  lemma  5,  £  x  fe[d'  ,d']? 

c(D,vk+l)  -C(D-i,vk)+9j(x). 

If  x<d',  let  6.  ■minid'  -x,,  _£  d,  -6].  Then  for  any  e30<e<6,,  x+e  is 
1  A  k+1  1  1 

in  the  domain  of  9j(*)  and  D  +  e  is  in  the  domain  of  C(» »v^+^) «  Furthermore, 

cpj  (x  +  e)  a9j  (x)  and  by  definition, 

c<D  +  e.vk+1)  sC[(D  +  £)-(i+e),vk]  +9^  (x  +  e). 


Therefore, 
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0* CCD  +  e.v^)  -C(D,vk+1)  *C(D  -x,^)  +9j(x+e)  -C(D  -x,vk)  +9j(x) 

-Tj(x  +  e)  -^(x) 

and  by  continuity  of  the  last  term  goes  to  zero  with  e.  If  x*d',  let 

6-  -mini.  ce.2  -  D, .  Jj.  d.  +d'  -  D} .  Note  that  if  d*  *  d  ,  2  d.  +d'  -  D 

2  lfeSlt+l  1  lfc5k  1  J  Lfe!>k  1 

*  i€Sk+1^i  ”  °  >0,  and  if  d'  ‘  iSk“i’  i€^kdi  +°  ‘  ®  *  i^J*!  "  iSfc^l  >0, 

therefore,  62  >0  and  for  any  e  3  0<t<62»  D  +  e  is  in  the  domain  of 
Furthermore,  e  *  .  Tt,  d,  +  d*  -  D  -  .  £  d.  ab  +  e-d',  hence  v,  is  feasible  for 
D  +  e  -x  and  by  definition  of  C(-,vk+^),  C(D  +  6,vk+^)  *  C(D  +  e  “  x,vk) +9^  (x) . 
Therefore, 

0*C(D  +  e,vk+1)  -  C (D» vk+1>  * C (D  +  «  -  i, ^  (x)  -  C(D-5c,v fc)  -  9j  (i) 

»C(D  +  e  -x,^)  -C(D  -x,^) 

and  tne  last  term  goes  to  zero  with  e  by  the  induction  hypothesis.  Therefore 
C(*,vk+1)  is  right  continuous. 

To  prove  left  continuity,  suppose  D  fc  (.  c_£  d  >.c  ^  d1-]  and  let 

op  ,  tfeSk+l  1  lfcSk+l 

(D  }.  ,  c  [,-„2  d, 2  d1]  be  any  sequence  such  that  D  to.  For  each  n 
1  “  1-1 

define  d^,  and  d^  as  d^  =»max{d^  d^}  and  d^  =*min[d^,Dn  -  ^  c^}.  By 

lemma  5  3  x^fd^.d^]  for  each  n  such  that 

Define  d'  -max[dj,D  -  d^}  and  d'  »min(dj,D  -  ^  d^}  and  for  each  n  define 

x  as  follows, 
n 


*n  "  S 


D  -  D  +x  if  d'  <D  -D  +x  <d' 
n  n  —  n  n 


if  D-D  +  x  s  d' 
n  n  — 

if  D-D  +x  ad' 
n  n 


;  .-yti 
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Then  for  each  n,  0*x  -x  iD-D  .  This  is  shown  by  considering 

n  n  n 

each  of  the  three  cases  defining  xn«  * 

Case  1.  x.  ■D-D.+ic.  *D-D.iO 
.  n  n  n  n 

Case  2.  x  *d'  and  D-D  +x  sd'.  From  the  definitions  of  dj,  d' 
— — — •  n  —  n  n  ~  *n 

and  the  fact  D-D  4  0  we  have 
n 


d'SD-D  +  x  ^d' 
— n  n  n  — 


-  &'  ■ 6  •  idjt  •“«  -  «lkV 

.'.D-D  id'  -  d'  >d'  *x  40,  since  x  4  d' . 
n  —  n  —  n  n  — n 

Case  3.  x  »d'  and  D-D  +  x  id'.  This  implies  directly 
-  n  n  n  r 

D-D  4d'-x  *x,  and  since  by  definition,  d*  4d  ,  then  d'-x  4d  -x  4  0. 
n  n  n  *  n  n  n  n 

Therefore,  as  shown  in  lenma  5,  x  is  in  the  domain  of  9.(*)  and  v,  is  feasible 

tv  j 

for  D-x  for  all  n  since  x  €[d'  d* ] .  Hence,  for  every  n, 
n  n  — 


°‘C<5-W  -c(VW±<c<5-vV+Ys»>i  ■c<».w-c<5-vv-vi«) 

+  c  (5  -  in,  vk)  -C(Dn-  x,,^)  +V.  (in)  -<Pj(Xn) 

‘  C<S-VV  -0<W2k>+Vi»>  -’’jV* 

where  the  last  inequality  results  from  lemma  5. 

Since  both  C(*,Vjt)  and  9j(*)  are  continuous  on  closed  intervals,  they 
are  uniformly  continuous.  Hence,  for  any  e>0  fiN3  Vn>N  the  last  term  is 
smaller  than  e.  Therefore,  C(D,vk+1)  -  C(DQ,vk+1) -0  and  the  lenma  is  proved. 

The  following  result  is  based  upon  the  assumption  of  convexity  for  the 
i*l,...,N,  rather  than  continuity  and  monotonicity.  Convexity,  of 
course,  implies  continuity.  Since  convexity  is  too  strong  of  an  assumption  for 
the  applications  to  be  considered,  this  result  will  not  be  used  in  the  sequel 
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and  is  included  merely  Co  indicate  an  alternative  development. 

Lengja_7.  If  <P^(»)  is  convex  on  [d^,d^]  for  i*l, ...,N,  then  for  any  v€[0,l)N 
and  S-^(v),  C(«,v)  is  convex  on  [Jjgd^jjjgdJ . 

Proof.  If  Isi  “1  the  lemma  is  trivially  true.  Suppose  for  some  kfc{l,...,N*l) 
the  lemma  holds  if  |s|  «k  but  3  Sk+L  6^  3  |sk+1l  -k+1  and  for  vfc+1  •J rl(Sk+1) 
C(*,vk+1)  is  not  convex.  Then,  2  D^,  D2>  Dj,  all  in  -i’ies^  ^i^  with 

D1<D3<D2  3 

C(D3»W  >a  C(D1*W  +  (1^)C(VW 

where 


Increasing  and  continuous*  Now,  assuming  a  set  of  N  subsystems  has  sufficient 
capacity  for  a  given  range  of  demand,  [D,D] ,  we  wish  to  consider  the  minimal 
cost  possible  using  any  feasible  v€{0,l}^  for  a  given  demand  D  £[D,D].  If  we 
define  the  set  of  feasible  v  for  a  given  demand  D,  V(D) ,  as 

V(D)  -  {v  6[0,1}N  |  v  if  feasible  for  D£[D,Dj}  (38) 

then  the  minimal  cost  for  meeting  D  with  any  collection  of  subsystems,  C  (D), 


is  defined  by 


C*(D) “v^fa)005’-5’  D€te»®J* 


The  following  results  characterize  the  function  C  (•)  which  is  not  necessarily 
monotone  increasing  or  continuous  on  all  of  [D,D] . 

Lemma  8.  If  for  the  interval  (D^>D2> c  VD£(D^,D2)  Hv^ 


G  (D)  -  0(0,^) 


and  for 


(Dl,D2>C[i©D^i,i§I)V’  (41) 

«Uf 

then  C  (•)  is  monotone  increasing  and  continuous  on  (D^,D2). 

Proof.  For  any  D',  D"€(D^,D2)  9D'  <D",  let  Vp,,  be  the  vectors  for  which 
(40)  holds  for  D'  and  D"  respectively,  and  let  S^,  ■•'(Vp,)  and  SD»  be 

the  corresponding  sets  for  which  (41)  is  satisfied.  Then  v^,  and  v^,,  are 
both  feasible  for  all  0  6(0^0^  and  C(*,v^,),  C(»,^)„)  are  monotone  increasing 
on  this  interval.  Therefore, 


C  (O') -C(D',vd,)SC(D,,vd,1)  <C(D",vd„) -C  (D") 

•fg 

implying  the  monotonicity  of  C  (•)  on  (D^,D2). 

Now,  for  5«;(D^,D2)  let  vg  and  Sg  *^(vg)  be  the  corresponding  vector 
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and  set  for  which  (40)  and  (41)  hold,  respectively.  Then,  from  the  definition 


of  C  (•). 


0*C*(D)  -C*(D)  *C(D,Vg)  -C(D,vg)  ¥D€(D,D2) 


and  the  last  term  goes  to  zero  as  D  i  D  by  the  continuity  of  C(»,v£).  Hence, 

C  (*)  is  right  continuous  on  (D^,D2). 

For  every  D€(D^,D)  let  Vp  and  SD  “^(vD)  be  the  corresponding  vector 
and  set  for  which  (40)  and  (41)  hold,  respectively.  From  the  definition  of 

"k 

C  (•)  and  its  monotonicity  on  (D^,D2), 

0 £ C*(D)  -C*(D)  Sctf.Vp)  -C^.Vp)  VDt(DL,D). 

Now,  since  [0,l}N  is  finite  and  C(*,vD)  is  continuous  ¥D€(D^,D),  for  any 

e  >0  S  6e  >0  a  6e  >0  3  ¥  0<6<6g,  and  V  DG(D^,D),  0(5,^)  -  C(D  -  6,^)  < e. 

~  * 

Therefore,  the  last  term  above  goes  to  zero  as  D  t D,  and  hence,  C  (•)  is 

continuous  on  (D^^). 

★ 

Lemma  9.  For  C  (•)  defined  in  (39), 

lim_  C*(D)  »C*(D")  exists  ¥  D€(D,D]  (42) 

D-D~ 


lim,  C(D)  -C*(D+)  exists  ¥  D€[D,D).  (43) 

D-D* 

Proof.  For  any  D€(D,D]  3  D’€(D^,D)3  the  interval  (D’,D)  satisfies  the  hypothesis 
of  lemma  8  since  [0,1}”  is  finite.  Since  C  ( • ) t  and  is  continuous  on  (D',D), 
the  limit  C  (D  )  exists  in  (42).  The  result  (43)  is  proved  by  a  similar  argue* 
ment. 


Lemma  10.  3  D^,  D2  €[D,D]? 


D1<D2  atld 


if  a  d€[d,d]3 


and  2  s  e  3 
N 


and  for  vfy  ^[S] , 


C  (D1)>C  (D2), 


D1  <  D  *  °2 


ilSk  “5 


C*(D)  -C(D,v) 


C*(D")  >  C*(D). 


Proof.  Sufficiency.  Let  D,  S  and  v  satisfy  (46)-(49)  and  let  D2  -D.  Then 
by  (49)  3  such  that  (44)  and  (45)  are  satisfied. 

Necessity.  Let  Dp  D2  satisfy  (44)  and  (45)  and  define  v^t[0,l}N  as  the  vector 
for  which  C*(D2)  - C(D2,v£)  and  let  S°»</(v£).  Define  If  Do<Dl 


we  have. 


C*(Dl)  SC(D1,v^)*C(D2,vJ)  -C*(D2) 


which  contradicts  (45),  Hence  D^D^Dj.  If  (48)  and  (49)  are  satisfied  for 

D«*D°,  v*v^,  the  lemma  is  proved.  If  (48)  is  not  satisfied,  proceed  with  step  A 

below,  if  (48)  is  satisfied,  but  (49)  is  not  satisfied,  proceed  with  step  B. 

Step  A.  (48)  does  not  hold  for  v-Vj,  D*Dj.  Hence  2  Vj+^6[0,l}N  3 

* ,„k.  _ ,  k  k+l.  _,_k  k. 

C  (Dj) -C(Dj,Vj  )<C(Dj,Vj).  (50) 

Let  Sj+1  ■<r*’(Vj+1')  and  define  Dj+1  »  dp 

f  <  ~k+l  ~  k+l 

If  (48)  and  (49)  hold  for  v  » Vj  and  D*D^  ,  then,  as  is  shown 

below,  the  lemma  is  proved.  Otherwise,  if  (48)  does  not  hold,  repeat  step  A. 

If  (48)  holds  but  (49)  does  not  hold,  define  Kj  “k+l  and  proceed  with  step  B. 


;  J  **«■?  i «■ 
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~  K4  ~  ^+1 

Step  B.  (49)  does  not  hold  for  D*Dj  but  (48)  holds  for  v-v^  .  It  is 
shown  below  that  D^<D^,  hence  3  Dj+^€(D^,Dj^ )d 


*  »  *  14 1 
C  (D.+1)  s  C  (D.J). 


o  c  3  N  ^  a 

This  follows  from  lemma  8.  Let  Vj+^  ci.0,1)  be  the  vector  5  C  (Dj+1) 

-  C(Dj+l.vJ+l)  and  define  S°+1  -*<vj+1) ,  and  D°+1  -  ^  V 

It  is  shown  below  that  if  (48)  and  (49)  hold^or  D“D°+^  and 

v*v°+^,  the  lemma  is  proved.  Otherwise,  proceed  with  Step  A  if  (48)  does  not 

hold,  and  with  Step  B  if  (48)  holds  but  (49)  does  not. 

**  k  k 

We  first  show  that  (46)  is  satisfied  by  D*Dj  for  defined  above. 

k  o  o  * 

Clearly,  DjSDj.  It  was  shown  above  that  <Dq  and  C(Dq)*C  (D2)*  Assume  that 

for  Dj  ,  D1<Dj  a«d  C(dJ)  s  C*(D2) . 

k+1  k 

If  (48)  does  not  hold,  it  is  clear  from  Step  A  that  Dj  SD^  which 

,  „  ,  k+1  k+1.  ,_k  k+1.  .  /c_.  _ .k+1  k+1  *  _  . 

implies  C(Dj  ,v\  )iC(Dj,Vj  ).  Then,  from  (50),  C(Dj  ,v^  )sC  (D2).  Now, 

k+1  k+1 

if  Dj  *  D^,  then  v^  is  feasible  for  and  we  have 

if  1*4,1  k  k+1 

C  (DL)  <C(D1,vj  t)s.C(Dj,vj  1)SC(D2) 


which  contradicts  (45),  therefore  D^<Dj  .  Suppose  Dj+^SD^,  then  since 
>D^»  by  definition,  v°+^  is  feasible  for  and  we  have 

C*(D1)*C(D1,vJ+l)<C(ij+1,v°+l). 

But  from  (51)  and  the  hypothesis  C*(Dj)*C  (D2)»  we  have  C  (D^)  <C*(D^.^) 
sc  C  (Dj  )  SC  (D2),  which  contradicts  (45).  Therefore,  Dj  defined  by  Step  A  and 
B  all  satisfy  (46)  and  by  definition  of  Sj,  (47)  is  also  satisfied  for  S  ■  S ^ , 
Finally,  it  is  easily  seen  that  at  each  step,  Sj+^  or  S°+^  can  not  equal  some 


previously  defined  S^.  Therefore,  since  is  finite,  the  algorithm  defined 


above  must  terminate  for  some  and  v^  which  satisfy  (48)  and  (49),  thus 
proving  the  Lenma, 

Lemma  II.  If  C*(*)  is  continuous  on  an  interval,  [D^^]  c  [D,D] ,  it  is  monotone 
increasing  on  that  interval. 

Proof.  This  is  a  direct  implication  of  the  previous  lenma  since  if  S  D', 

“ft 

D"  D'  <D"  and  C  (D1)  >C  (D“),  there  is  a  discontinuity  between  D'  and 

D"  of  the  type  (49). 

Lemma  12.  If  for  D  (  (D,Dj  2v£lO,ljN  and  S»v(v)3 


C*(D)  =  C(D,v) 

(52) 

2  d.  <  D  s  2d. 
i€S  i€S 

(53) 

then 

C*(D)  »C*(6”).  (54) 

Proof.  Suppose  for  D  (  (D,D] ,  v  and  S  ®^(v),  (52)  and  (53)  are  satisfied.  Then 

for  D  ([  2d.  ,6) , 
i€S~ 

C*(D)  *C(D,V)  JCC(D,v)  =C*(D). 

,.C*(D  )£C*(D).  Suppose  C*(D>C*(0  )*a>0.  Clearly  2  D’6[D,D)3  for  the  interval 

—  * 

(D',d)  the  hypothesis  of  lemma  8  are  satisfied  and  hence  C  (•)  is  monotone  in¬ 
creasing  and  continuous  on  (D*,D).  Then, 

C*(D)  -C*(D)  a  a  VDt(D',D).  (55) 

*  — 

Let  be  the  vector  for  which  C  (D)=C(D,_^)  for  D£(D*,D).  By  the  choice  of 

O’  we  can  select  v^  so  that  it  is  feasible  on  the  entire  interval  [D',D].  By 

the  continuity  of  C(*,v)  for  each  v€l0,lj  ,  there  exists  6  >0  such  that 

—  —  or 
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OiC(D,v_)  -  C(D  -  6,v  )  Sa/2  when  0  <6  *6  . 

~v  —v  a 

But,  C*(D)  s  C  (D,  Vq)  V  D6[D',D],  so  for  D*D-6  where  0  <6  SminU^.D  -  D ' }  wfe~ 
hav® 

0 s  C*(D)  -C*(D)  sC(D,vd)  -C(D,vd)  ia/2 

which  contradicts  (55).  Therefore,  a«0  and  (54)  is  proved. 

Lemma  13.  If  for  D  6  [D,D)3  v  fctO, lj^  and  S 

C*(D)  -  C(D,v)  (56) 

and 

Ei,s»<  Ed,  (57) 

i€i>  i€S  1 

then 

C*(D)  *  C*(fT*" )  (58) 

Proof.  For  5,  S,  v  defined  above  assume  (56)  and  (57)  hold.  3  D’t(D,D]  such 
that  the  hypothesis  of  lemma  8  are  satisfied  for  the  interval  (D,D').  Hence, 

C  (•)  is  monotone  increasing  and  continuous  on  (D,D’)e  Furthermore,  for  each 
D  6  (D,D')3  Vp  €{0,lJN  9  C  (D)=C(D,vD)  and  v^  is  feasible  on  the  entire  interval 
[D,D'J.  Therefore, 

C*(D)  *  0(5,^)  *0(0,^)  -C*(D)  V  D  6(5, D ' ) 
which  implies  C* (5)  * C* (D*) . 

Now,  c[D,  £_d^]  j  c  (D)sC(D,v)  and  since  C(D,v)  iC  (D)  as  D  JD  we  have 
C  (D1’)*C  (D).  Therefore  C  (D)*C  (Er)  ,  proving  (58). 

The  following  theorem  incorporates  the  above  results  to  characterize 
C  (•)  on  the  entire  range  [D,DJ* 
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Theorem  3.  If  C  (•)  is  discontinuous  at  D  t [D,D]  then  the  discontinuity  is  one 
of  the  following  two  types. 

Type  1.  2  with 


and 


5. 4- 


icS 

i 

C*(D)  -  C  (D,  v^)  , 


C*(D“)  >C*(D). 


(59) 

(60) 

(61) 


and 


Type  2.  3  S2  6^  with  v£  (S2)^ 

D  *  2  d  , 

its2 

C*(D)  »  C(D,v2) 


(62) 

(63) 


C*^)  >C*(D). 


(64) 


VC  m 

Furthermore,  C  (•)  is  monotone  increasing  on  any  interval  in  [D,D]  which 

* 

contains  no  points  of  discontinuity  of  C  (•)• 

Proof.  The  final  statement  is  merely  lemma  11.  Suppose  Dt(D*D)  is  a  point  of 

discontinuity  but  neither  (59)  and  (60)  or  (62)  and  (63)  hold  at  D.  Then 

clearly  D  is  contained  in  an  interval  which  satisfies  the  hypothesis  of  lensna  8, 

which  implies  C(>)  is  continuous  at  D.  Therefore,  if  D  t (D,D)  it  satisfies 

either  (59)  and  (60)  or  (62)  and  (63).  Choose  £>0  such  that  D  is  the  only 

points  of  discontinuity  on  (D-s,D+£).  This  is  possible  since  3e^,£2  >0? 

the  intervals  (D -e  ^,D)  and  (D,D+s2)  satisfy  the  hypothesis  of  lemma  8,  therefore 
★ 

implying  C  (•)  is  monotone  increasing  and  continuous  on  these  intervals.  If 
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•fc  ^  —  -  1,  >v 

C  (D  )  >C  (U) ,  lemma  10  implies  (59)  and  (60)  are  satisfied  at  D  and  by  lemma 
13,  cV)-C*(D).  so  (61)  is  also  satisfied. 

If  C*(D  )<C*(D+),  the  discontinuity  cannot  satisfy  (59)- (61) 
simultaneously  since  lemma  13  implies  that  if  D  satisfies  (59)  and  (60), 

'ic  *  ^ 

C  (D)=*C  (D  ),  so  (61)  cannot  be  satisfied.  Suppose  (62)  and  (63)  are  satisfied. 
Then  lemma  12  implies  C  (D  )  * C  (D),  therefore  the  discontinuity  is  of  type  2. 

~  *  ~4-  *  ~+  * 

Finally,  if  D  =D,  then  only  C  (D  )  is  defined.  Suppose  C  (D  )  <C  (D), 

then  choose  £  3C  (•)  is  continuous  on  (D,D  +  s)  and  2 D' £(D,D +e)3  C  (D)>C  (D1). 

"ic 

Then,  by  lemma  10  3D"fc(D,D')  which  is  a  point  of  discontinuity  for  C  (•),  but 

,fc 

this  contradicts  the  definition  of  6.  since  D"  <D  +  e.  Therefore,  C  (D  )  >C  (D) 
and  hence  the  discontinuity  must  be  of  Type  2  since  if  D<i^sdi  ^  S3  for 
v  ^(S),  C  (D)=C(D,v),  C  (D^)=C  (D)  by  lemma  13.  The  fact  that  if  D  *D 
then  the  discontinuity  is  of  type  1  is  shown  by  assuming  C  (D  )  <C  (D) .  If  (62) 
and  (63)  are  satisfied,  (59)  and  (60)  must  hold  for  D  and  one  can  show  that 

'k  "fc  — ■  ^  ^ 

C  (D  )  <C  (D)  implies  2  v  3  C(D,v)  <C  (D),  a  contradiction.  Therefore, 

C  (D  )  >C  (D)  and  (59)  and  (60)  must  hold.  Q.E.D. 

4.  OPTIMAL  CONTROL  OF  THE  OFF-LINE  SUBSYSTEMS 
4.1.  Results  for  General  System  Dynamics 

As  discussed  in  the  general  problem  formulation,  if  the  coordinator's 
control,  Vj_(t),  for  the  ith  subsystem  switches  from  one  to  zero  at  t^  and  then 
switches  back  to  one  at  C2>t]_>  the  l°cal  control  for  that  subsystem  is  chosen 
to  minimize  J^^-t^.uS  given  in  (11)  subject  to  u^tU*"  and  the  state  differen¬ 
tial  equation  and  end  conditions  given  in  (12).  It  is  reasonable  to  assume  that 
during  the  off-line  interval  the  subsystem  could  be  maintained  at  the  operating 
steady-state,  since  this  steady-state  is  maintained  for  on-line  operation.  Hence, 


(65) 


it  is  assumed 

O-f^x^uSo),  i-l,...,N. 

The  coordinator  is  then  free  to  select  any  interval  of  time  for  a 
subsystem  to  be  off-line  since  assumption  (65)  implies  there  is  at  lease  one 
control  for  which  the  end  conditions  in  (12)  are  satisfied.  It  will  be  shown 
in  the  following  the;  lor  the  particular  models  of  interest,  the  minimizing 
control  exists  '  nen  it  is  known  there  exists  at  least  one  control  satisfying 
the  tpbv  problem  of  (12). 

Since  the  f^(x^,u^,0)  and  L^(x^,u^)  are  independent  of  time  for  every 
i=l,...,N,  the  properties  of  the  optimal  control  problem  for  the  off-line  sub¬ 
systems  will  be  derived  from  the  following  generic  model. 

Consider  a  system  described  by  the  differential  equation 

x(t)  =  f(x(t),u(t))  (66) 

where  x(t)  £Rn  is  the  system  state,  u(t)  tRm  is  the  control  variable,  and 
f  :  Rn+m  —  Rn  is  continuous  with  continuous  first  partial  derivatives  WRT  the 
components  of  x.  Then  for  piecewise  continuous  u  and  a  given  x(0),  it  is  well- 
known  there  exists  a  unique  continuous  solution  to  (66).  It  is  further  assumed 
that  for  some  compact  set  UCRm,  u(t)  must  satisfy 

u(t)€U  V  t  £[0,T]  (67) 

and  that  the  solution  to  (66)  must  satisfy  the  boundary  conditions 

x(0)*x  and  x(T)=x  (68) 

for  a  given  xSRn.  The  control  is  then  selected  to  satisfy  (66)  through  (68) 


while  minimizing 
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J(T,u)  -  L(x,u)dt  (69) 

where  L  :  Rn+m-*R^  is  a  non-negative  function.  Finally,  assume  3  u  6U  such 
that 

0  *  f(x,u).  (70) 

Now,  if  a  piecewise  continuous  control  exists  which  satisfied  (66) 

through  (6S)  and  minimizes  J(T,u)  in  (69),  let  the  minimal  value  be  indicated 
* 

by  J  (T)  given  as 

J*(T)  »minf*  L(x,u)dt,  (71) 

utU 

where  the  minimization  is  subject  to  the  above  constraints. 

Of  particular  interest  to  the  coordinator  is  how  the  cost  of  the  off¬ 
line  controls  varies  as  a  function  of  the  off-line  time  interval.  Therefore, 

in  terms  of  the  generic  optimal  control  problem  defined  above,  we  wish  to 

* 

determine  the  characteristics  of  the  function  J  (T)  given  in  (71).  We  first 

* 

give  some  results  concerning  the  special  case  when  J  (•)  is  monotone  increasing 
and  the  state  equation  is  of  the  general  nature  of  (66).  Then  the  linear-time 
invariant  case  of  a  particular  "fuel  optimal"  problem  is  considered  leading  to 

.j* 

formulas  for  the  computation  of  . 

* 

Lemma  14.  If  J  (T)  given  by  (71)  is  monotone  increasing  for  T  6 (T^jT^) ,  then  it 
is  continuous  for  all  T^T^jTj),  differentiable  almost  everywhere  on 
^T1,T2^  ancl  t*ie  derivative  exists  at  TS^^), 

* 

Oi-f  I  ,  *  L(x,u).  (72) 

0  x»x 

J  (T)  *  J*(T  +  i)  SJ  (T)  +  d  •  L(x,u),  hence,  taking  the  limit  as  A-0 


Proof. 
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gives  right  continuity.  Suppose  lim  J  (T-A)-J  (T  )  <J  (T).  Let 

A— 0"*" 

a»J  (T)  -J  (T'  ')  >0  and  select  A  >0such  that  L(x,u)*  A  <or.  Then 

J*(T)  s  J*(T-A)+A.  L(x,d)<J*(T("))  +  a-  J*(T), 

which  is  a  contradiction,  proving  left  continuity.  Differentiability  a.e. 
follows  from  a  theorem  of  real  analysis ([26] ,p.  76).  Finally,  for  all  A>0 
it  is  clear  that 

*  * 


and 


0  *  +*>-->  rn  s 

A 


o  ‘  JV-;w*a) «  L&u'). 


proving  (72). 

* 

The  following  two  lemmas  establish  conditions  under  which  J  (T)  is 
in  fact  monotone  increasing. 

For  x£Rn,  define  J*(x)  and  J^Cx.T) 


*  ~  fT 

J.(x)  *  min  L(x,u)dt 
U»T  ° 

where  the  minimization  is  subject  to  (66),  (67)  and  the  end  conditions 


(73) 


x(0)  -X,  x(T)  -  X, 


(74) 


and 


*  ~ 


nT 


J,  (x,T)  -  min  J  L(x,u)dt 
C  u  ° 


(75) 


where  the  minimization  is  subject  to  (66),  (67)  and  the  end  conditions 


x(0)  ■  x,  x(T)  -  x. 


(76) 


Lemma  15.  For  J  (T)  in  (71),  J  (T)  is  monotone  increasing  WRT  T  whereever  it 


■  i  •  *<»*•  *• 
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exists  if 

J*(x)  +  J*(x,T)  a J*(T)  (77) 

^  'ff  rj 

for  all  x  and  T  such  that  J^(x)  and  J2(x,T)  of  (73)  and  (75),  respectively, 
both  exist. 

Proof.  Necessity;  Suppose  J  (T)f  and  St',  x1  3 

J*(x')  +J*(x',T')  <  J*(T') 

•/f  ~ 

Let  T  (x')  be  the  minimal  minimizing  time  corresponding  to  the  minimizing  control 

^  M  *fc 

and  trajectory  for  J^(x').  Then  T  (x')  >0  and,  from  the  definition  of  J  (•)» 

J*[T'  +  T*(x')]*  J*(x')  +J*(x',T'), 

which  implies  J  [T'+T  (x')J  <J  (T*),  contradicting  the  monotonicity  of  J  (•)• 

Sufficiency:  Suppose  (77)  holds  but  2  0<T1<T2  9  J*(TL)  >J*(T2).  Let 

X  ( t , T2 ) ,  u  ( t , T2 )  be  the  optimal  control  and  trajectory,  respectively,  corres- 
it  ' — it  ic 

ponding  to  J  (T2)  and  define  6  *  T2  -T^.  If  x  4x  (6,T2),  then  the  following 
relations  hold; 

X 

J*^*,^)  «/  2  L[x*(t,T2),u*(t,T2)]dt  (78) 

6 

which  follows  from  the  principle  of  optimality  and  the  time  independence  of 
f(x,u)  and  L(x,u); 

J*(x*)*/  L[x*(t,T2),u*(t,T2)]dt  (79) 

o 

*  — 

which  follows  from  the  definition  of  J^(x). 

Summing  (78)  and  (79)  gives 

J*0O  +  J*^*,^)  s  J*(T2)  <*{1^, 
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which  contradicts  (77),  proving  the  leinna. 

For  the  special  case  when  the  state  is  a  scalar  we  have  the  general 

result: 

1  * 

Lenma  16-  If  xtR.  ,  then  J  (T)  of  (71)  is  monotone  increasing. 

Proof.  By  lemma  15,  the  lemma  is  false  if  2  x',  T  3 

J*(x)  +  J*(x,T)  <  J*(T) .  (80) 

Assume  such  an  x  and  T  exists  and  let  u^(t),  x^(t)  denote  the  optimal  control 

and  state  trajectory  corresponding  to  J^(x)  and  let  be  the  corresponding 

minimal  minimizing  time.  Let  u2(t),  Xj(t)  be  the  control  and  state  trajectory 

corresponding  to  J*(x,T).  We  show  that  (80)  implies  there  exists  a  control 

* 

satisfying  (67)  and  (68)  for  which  the  cost  is  less  than  J  (T). 

Case  1:  3  t  6  [0,  ala(T,TI)]^  xL(t)  -Xg  (t). 

Let  a  concrol,  u(t),  be  defined  by 

u^t)  OSt^t 

u(t)  =  < 

lu2(t)  t<t*T. 

Then  the  state  trajectory  generated  by  u(t)  from  (66)  with  x(0)  »x  is  given  by 


x(t)  which  satisfies: 


xL(t)  0stst 


x2(t)  t  <  t  s  T, 


since  xx(t)  satisfies  (66)  for  u^t)  and  x£(t)  satisfies  (66)  for  u£(t).  Now, 
x(0)  -xL(0)  -x  and  x(T)  «x2(T)  *x.  Furthermore, 


T  t  T 

J  L(x,u)dt  *  j*  L(x1,u1)dt L(x2,u2)dt 


*  J*(x)  +J*(X,T), 


'  ‘4  ***  1 


so  by  (80),  f  L(x,u)dt <J*(T). 
o 

Case  2;  x^(t)  and  x2(t)  do  not  intersect. 

First  note  that, 

V  t  €(0,1^,  xL(t)  (t(x,xj,  (81) 

since,  if  for  some  such  t  x^(t)  «x  or  x^(t)*x,  there  would  be  a  time  interval 

shorter  than  T^  for  which  a  control  and  state  trajectory  could  satisfy  (66), 

*  ~ 

(67)  and  (74)  with  cost  less  than  or  equal  to  J^(x),  and  this  would  contradict 
the  definition  of  T^. 


Secondly,  we  note 


tl  <T. 


Suppose  (82)  were  note  the  case,  i.e.  T^iT.  If  x>x,  then 
x^(0)  -x<x2(0)  »x  and  by  the  continuity  of  x^(t)  and  x2(t)  and  the  fact  they 
do  not  intersect,  x^(t)<Xj(t)  V  t  t{0,T].  In  particular,  x^(T)  <x2 (T)  « x, 
which  implies  3  t  €[T,T^)3  x^(t)»x,  since  x^(T^)  >x^  which  contradicts  (81), 
if  T^>T.  If  T^  ■  T  we  have  x^(T)  *x,<x  contradicting  the  assumption.  For 
x<x  a  similar  contradiction  is  reached.  Finally,  if  x»x,  then  T^»0  and 

^  A  •ff  mm  *j|p 

J1(x)»0  implying  J2(x,T)  »J  (T),  contradicting  (80).  Therefore,  (82)  is  true. 
Thirdly,  it  must  be  the  case  that  3t€[0,T-T^]3 

x^t)  -x2(t  +  T1).  (83) 

To  prove  (83),  assume  x>x  and  suppose  x2 (t)  +  x2 (t  +  T^)  Vt€[0,T-T^]. 
Then,  either  x2  (t)  >x2  (t  +  Tj.)  or  x2  (t)  <x2  (t  +  T^)  V  such  t,  and  since 
x^O)  «x>x1(0) -x,  by  assumption,  x2(t)  >x^(t)  t^O.T^].  In  particular, 

x^T^)  >x^(Tj^)  «x  and  therefore,  x2  (0)  ■  x  <x2  (0  +  T^) ,  which  implies 


4 .;*,+’*** 
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for  Che  very  restricted  class  of  optimal  control  problems  to  be  considered 

below  it  is  not  true  that  J  (T)  is  necessarily  monotone  increasing  which  means 

*  dJ* 

other  approaches  are  needed  to  analyze  the  properties  of  J  (T)  and  -gj-. 

Linear  Time  Invariant  Systems 

Consider  the  linear  autonomous  system  state  equation 

x  ®  Ax  +  Bu  (85) 

and  let  the  control  constraint  set,  U  be  given  by 


U  *  {u  €  Rm  I  0  &  u^  *  u^,  u^>0,  i»l,...,m}.  (86) 

For  controls  satisfying  (67)  with  U  given  by  (86)  and  solving  the  tpbv  problem 
(85)  and  (68)  for  a  given  xfRn,  let  the  cost  be  given  by 

T 

J(T,u) -J  k'udt,  k6R®  (87) 


where 


k.  >0,  ial,..»,m,  k ■  [k. , . . . ,k  ] ' . 


This  is  a  type  of  "fuel  optimal"  problem,  if  the  components  of  the 
control  are  considered  rates  of  fuel  consumption  with  unit  costs  given  by  the 
k^,  i«l,...,m.  The  properties  of  the  optimal  solution  to  this  problem  are 

considered  in  the  sequel.  For  now,  assume  for  T“T  there  exists  an  admissible 

♦  A  *  ♦  A 

control,  u  (t,T),  for  Q*tsT,  and  corresponding  state  trajectory,  x  (t,T), 

satisfying  (85)  and  the  boundary  constraints  (68)  for  which  J[T,u  (t,T|»J  (T) 
the  minimal  value  of  the  cost  (87). 

It  is  well-known  from  the  maximum  principle  that  there  exists  a 
costate  trajectory,  ,\  (t,T),  t  S[0,T],  which  satisfies  for  some  \  (I)tR  , 


j  'Jut,.**-.. 


X  (t,T)  -  -A'X  (t,T) 


(89) 


with  initial  condition 


X*(0,T)  »X*(T) 


(90) 


such  that  for  i»l,...,m. 


u*(t,T) 


if  b|X  (t,T)+ki>0 


if  (t,T)+ki<0 


where 


(91) 


B»[b  *  b  *  ...  I  b  ],  b.  6Rm,  i-l,...,m. 
x  •  ^  •  •  m  x 


(92) 


For  now,  assume  (91)  determines  u^t.T)  uniquely  a.e.  on  [0,T].  Now,  suppose 

A 

an  optimal  control  can  be  found  for  all  T  in  some  neighborhood  of  T,  with  the 
optimal  control,  state  trajectory,  costate  trajectory  and  costate  initial 

'it  'ff 

condition  denoted,  for  each  T,  by  u  (t,T),  x  (t,T),  X  (t,T),  Xq(T)  respectively. 
★ 

Hence,  J  (T)  is  defined  in  a  neighborhood  of  T  and  formally  from  (87)  we  have 


I  A  -  k'u*(T,T)  H-J T  k'  |  „  dt, 

T-T  o  T=*T 


(93) 


where  (93)  is  an  application  of  Leibniz'  rule.  In  the  following  an  explicit 

form  for  (93)  is  derived  and  sufficient  conditions  are  given  for  its  validity. 

Towards  that  end,  make  the  following  definitions.  For  X  6Rn  let 

o 

X(t,XQ)  be  the  solution  to 


X(t,XQ) --A'X(t,Xo),  taO, 


(94) 


with  X(0,X  )  *  X  . 

o  o 
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Define  for  taO,  i«l,...,m,  f^(t,XQ)  as 

ft(c,X0)  -b*  K(t,X0)+ki  (95) 

and  let  u^tjX^)  be  given  by 

o  if  fi(t*A,o)i0 

,  i«l, (96) 

ui  fi(t»^o)<0 

Let  u(t,X  )  » [u, (t,X  ),..., u  (t,X  )]'.  Finally,  define  x(t,X  )  as  the  solution 
o  l  o  m  o  o 

for  t  a  0  of 

x(t,XQ)  *  A  x(t,XQ)  +  B  u(t,XQ)  (97) 

with  x(0,Xq)  *x. 

Now,  define  F  :  R1*^  “*Rn  by 

F(T,Xq)  »  x(T,Xo)  -  x  (98) 

*  *  ^ 

and  note  that  for  T  =  T  and  X  »X  (T)  as  in  (90)  we  have 

o  o 

F[T,X*(T)]  -  0.  (99) 

,  * 

The  general  conditions  for  which  it  is  guaranteed  X  (t)  is  a  continuous  function 
of  T  in  a  neighborhood  of  I  and  a  formula  for  its  derivative  are  to  be  derived 
using  the  following  implicit  Function  theorem. 

Theorem  4.  Let  F^  [T,X  ]  denote  the  derivative  of  F[T,X  ]WRT  X  evaluated  at 
o 

(T,X0)  and  FT(T,Xo)  denotes  the  derivative  WRT  T.  Then,  if  for  a  given  (T,Xq) 
we  have, 

F(T,Xq)-0,  (100) 


F(T,X  )  is  continuous  in  a  neighborhood  of  (T,X  ), 


(101) 
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F.  (T,X  ),  F_(T,\  )  are  both  bounded  and  continuous  in  a 
A  O  I  o 
o  ^ 

neighborhood  of  (T,Xq)  (102) 

F(T,Xq)  is  differentiable  in  a  neighborhood  of  (T,^),  (103) 

and  [F^CT,^)]-1  (104) 

then 

2  G  :  R1  -  Rn  3 


A  A 

G  is  continuous  in  a  neighborhood  of  T,  denoted  by  ?1(T),  (105) 

F[T,G(T)]  -  0  for  all  T  in  ??(T),  (106) 

£  *  G(T)  and  (107) 

dT^T*T  *”^X  i!T^’^o^*  (108) 

o 

Proof,  pages  194-198  of  [13]. 


To  apply  theorem  4  we  first  analyze  some  properties  of  the  optimal 
control  problem  being  considered  and  then  state  specific  conditions  under 
which  the  optimal  solution  leads  to  a  satisfaction  of  the  hypothesis  of 
theorem  4.  It  is  also  shown  that  the  G(T)  in  (105)  actually  equals  Xq(T)  in 
the  entire  neighborhood,  7;(T). 

Consider  again  the  equations  (68)  and  (85)  through  (88)  defining  the 
optimal  control  problem.  The  following  results  characterize  the  optimal 
solution  and  most  of  the  proofs  will  be  omitted  since  the  arguments  are  similar 
to  known  results  concerning  a  somewhat  different  fuel-optimal  control  problem 
in  [14],  We  first  make  the  following  standard  definition: 

Definition:  If  for  an  optimal  solution  to  (85)-(88)  which  satisfies  (89)-(91) 
there  is  for  some  i  €[l»  •  • .  an  interval  in  [0,T]  such  that  b£X  (t,T)+ki»0 


on  Che  entire  interval,  then  the  problem  is  called  singular. 

Since  we  wish  for  (91)  to  uniquely  determine  the  optimal  control _ 
almost  everywhere,  it  is  desirable  to  be  assured  the  problem  is  not  singular. 
The  following  are  sufficient  conditions  for  this  to  be  the  case. 

Lemma  17.  If  A  is  nonsingular  and  the  pairs  (A, b^)  are  controllable  for  all 
1*1,..., m,  then  the  optimal  control  problem  (85)- (88)  is  not  singular  for  any 
T. 

When  A  and  B  of  (85)  satisfy  the  hypothesis  of  lemma  17  the  problem 
will  be  called  "normal"  (note  the  usual  definition  of  a  normal  problem  does 
not  include  the  nonsingularity  of  A,  however,  for  convenience  in  the  following 
it  will  be  included).  The  following  lensnas  deal  with  the  existence  and 
uniqueness  of  an  optimal  control  for  a  normal  problem  of  the  type  in  (85) -(88). 
Lemma  18.  An  optimal  control  minimizing  J(T,u)  (87)  exists  if  there  exists  at 
least  one  control  in  the  constraint  set  of  (86)  for  each  tc[0,T],  such  that 
the  tpbv  problem,  (68)  and  (85)  is  satisfied. 

Proof,  pp.  127  ff.  in  [15]. 

From  the  general  assumption  (70),  lenraa  18  implies  that  for  any  time 
interval  an  optimal  control  exists.  Note,  lemma  18,  did  not  require  a  normal 
problem. 

Lemma  19.  If  the  problem  (85)-(88)  is  normal  and  an  optimal  control  exists, 
then  it  is  unique. 

Proof.  Similar  to  proof  of  Theorem  6-14  in  [14]. 

Note  that  by  uniqueness  one  means  up  to  a  set  of  measure  zero  since  changing  a 
control  at  isolated  points  affects  neither  the  solution  to  (85)  nor  the  value 
of  the  integral  (87). 


8 


Lemma  20.  For  a  normal  problem  given  by  (85)-(88)  if  a  control  exists  with 
corresponding  state  and  costate  trajectories  satisfying  (68)  and  (90)-(91) 
for  some  costate  initial  condition,  then  the  control  is  unique  in  that  for  no 
other  control  which  solves  the  tpbv  problem  (85)  and  (68)  can  (89)-(91)  be 
satisfied  for  any  costate  initial  condition. 

Proof.  Similar  to  proof  of  Theorem  6-15  in  [14], 

We  now  apply  these  results  to  an  analysis  of  F(T,Xq)  and  the  condi¬ 
tion  (99). 

Lemma  21.  For  a  normal  control  problem  given  by  (85)-(88),  if  for  T>0  there 
exists  a  X.o€Rnsuch  that  F(T,Xq)»0,  where  F(T,Xq)  is  defined  by  (94)  to  (98), 
then  u(t,XQ)  of  (96)  is  the  unique  optimal  control,  for  T*T. 

Proof.  From  the  definition  of  F(T,Xq)  and  equations  (94)  to  (98)  it  is  clear 
that  if  F(T,Xq)=0,  then  £(t,XQ),  u^(t,&Q)  anc*  x(t,XQ)  satisfy  the  necessary 
conditions  (89) -(91)  for  an  optimal  control  as  well  as  the  tpbv  problem  (85) 
and  (68),  for  T  =  T.  Hence,  by  leranas  19  and  20  u(t,XQ)  is  the  unique  optimal 
control  for  T  =  T. 

Lemma  22.  F(T,X  )  is  continuous  for  all  X  (Rn.  iaO. 

—  ■  o  o 

Proof.  This  follows  from  the  definition  of  F(T,Xq)  and  the  continuity  of  the 
solution  of  a  system  of  differential  equations  with  respect  to  the  initial 
conditions . 

3x(T,Xq) 

We  now  evaluate  [T,  XoJ  *  3X  — ,  following  the  work  of  Stefenak 

o  o 

[16]  on  the  sensitivity  of  solutions  of  differential  equations  with  discontinuous 
elements.  Define  the  matrices  A(t,XQ)  and  X(t,XQ)  as 

dx(t,X  ) 

.  .  .  .  A  O 


(109) 
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oX(t,\  ) 
A(t,X0)  ■  S\ 


(HO) 


where  X(t,\Q)  and  are  defined  by  (94)  and  (97),  respectively.  Their, 

from  (94),  A(t,XQ)  is  the  solution  to  the  matrix  differential  equation 


with 


A(t,\o)  -  -  A'  A(t,\Q) 


A(0,\q)  -  1. 


(Ill) 


The  differential  equation  for  x(t,/\Q)  involves  piecewise  constant 
control  values.  Consider  the  set  of  switch  points  for  u(t,\Q)  on  [0,^°), 

T(\q)  defined  by 

T(\q)  -  {tt[0,=°)  I  ui(t",XQ)  j‘ui(t+,Xo)  for  some  it(  1, . . .  ,mj  }  .  (112) 

For  a  normal  control  problem,  7 (X  )  is  a  set  of  isolated  points  on  [O,00). 


Define  N(T,Aq)  as  the  number  of  points  in  T(X.q)  less  than  or  equal  to  T.  Then 


we  can  define 


»<T.V 

v-1 


T(Xq)  i)  [ 0, T] 


(113) 


where 


0S  C1  <"*  <  *"N  (T,Xq)  st> 


(114) 


Define  Y  :  Rn+i  -<?[  { 1, . . .  ,m]  ]  as 


Y(t,AQ)  »  ti€(l,...,mj  I  ui(t",Xo)  j‘ui(t  , ) } ,  (115) 


indicating  the  components  of  u(t,XQ)  which  switch  at  t.  Note  that  if 

t  jfc j” ( \ q )  then  Y(t,\Q)*£.  Now,  the  differential  equation  satisfied  by 

X(t,X  )  between  switch  points  of  u(t,X  )  is 
o  0 
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X(t,X  )  =  A  X(t,X  ) . 


(116) 


Clearly,  since  x(0,Xq)  =  x  for  all  Xq  €r  , 


X(0,X  )  =  0 

o  nxn 


(117) 


At  each  point  tv6J(Xo)  there  is  a  jump  inX(t,XQ)  which  is  given  by 


X(t  ,X  )  -  X(t  ,X  )«1£V  2  ,  .  b .  [u.  (t  ,X  )-u.(t  ,X  )] 

v  o  v  o  J-£Y(t^,Xo)  il  v*  o  j.  v  o/J 


dta 
_ } 

dX 


(118) 


where  is  the  sensitivity  of  the  switch  time  for  the  ith  component  of  the 
o 

control.  This  is  computed  by  applying  the  implicit  function  theorem  (theorem 
4)  to  the  switching  functions  fi(t,XQ),  i=l,...,m,  of  (95). 

Lemma  23.  For  the  switch  time  t^€T(T,Xo),  and  i£Y(t^,Xo)  if 


b'A«  X(tv,Xo)  *0 


(119) 


then,  the  switch  time  sensitivity  of  the  ith  component  of  the  control  with 
dt* 

respect  to  X  >  ,  is  given  by 

O  OA 

o 

A'<'v.Vbl 


6X  "  b.'  A'  X(t  ,X  )  ‘ 

O  1  V  O 


(120) 


Proof.  From  (95)  and  the  definitions  of  t.,,  and  Y(t  ,X  ),  f.(t  ,X  )*b?X(t.X  )+k.  =0 
-  v  v’  o'j  iv  v1  o'  i  v  ’  o'  i 

°^i  ~ 

Clearly,  fi(t,XQ)  is  continuous  and  differentiable.  -^r*  *■  -b£ A  'X(t,X0)  is 

df  i 

continuous,  bounded  and  invertible  by  (119).  Furthermore,  T^-A^t.X  )b. 

OA  O  1 

O 

is  also  continuous  and  bounded.  Therefore,  theorem  4  implies  (120). 

Combining  (116),  (117),  (118)  and  lemma  23  gives, 

Lemma  24.  The  sensitivity  of  x(t,XQ)  with  respect  to  Xq  is  given  by  X(t,XQ) 
by  (121)  and  (122)  below  for  all  t  )£ T(Xq) ,  if  for  all  t^t  T(X  ),  bj  A' X(tv,XQ)  *  0 
when  i  €Y(tv,XQ). 
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N(t’Xo)  A(t-t  ) 


X(t.Xo)-v£i  e  V'[X(t+Xo)-X(t;,\0)J 


(121) 


where 


a. (t+,X  )  -  u. (t  ,k  ) 
I'v7  i  v*  o7 


X(tv*\>*  "X(tv,Xo)  aitY(tv,Xo)  A*  5C(tv,Xo)  bibiA(tv’\P  *  (122) 


Proof.  (122)  follows  from  (118)  and  lemma  23,  and  (121)  is  the  result  of 
integrating  (116)  subject  to  (117)  and  (118). 

Conditions  for  which  theorem  4  may  be  applied  to  the  fuel  optimal 
control  problem  can  now  be  stated. 

Theorem  5.  If  the  control  problem  (85)  to  (88)  is  normal  and  for  T>0, 

k  * 

Xq(T)  is  the  initial  condition  for  a  costate  trajectory  corresponding  to  an 
optimal  control  and  optimal  state  trajectory  satisfying  (68)  and  (89)  to  (91), 
and,  provided  the  following  conditions  are  satisfied; 


T  £x[X*(t)J  » 

for  all  tv€J[A*(T)]n[0,T]  and  i  *Y[tv,\*(T)l , 


k  * 


bjA'£[tv,X0(T)]  *  0 


(123) 


(124) 


.  k  * 


and  for  X[t,A  (T)]  given  by  (121) 


X[T , (T) 3  is  non-singular. 


(125) 


then,  there  exists  a  continuous  function  \q(T)  defined  in  a  neighborhood  of 

T,  7((t) ,  such  that  the  control  uft,A  (T)]  generated  through  equations  (94)-(96) 

ax*(T) 

is  the  unique  optimal  control  for  T  €^J(X)  and  — ^ -  evaluated  at  T  is  given  by 


°  1  -  t“*-XCT,X*(T)| i-1  •  (Ax  +  Bu[T,\*(T)]j 


dT  T  *  T 


(126) 


a  k  * 

Proof.  We  show  that  for  F[T,X  (T)J,  the  hypothesis  (100)-(104)  of  theorem  4 
are  satisfied.  Clearly  F[T,\*(T)]  a0  since  X*tT>^*(T) ]  »x.  (101)  holds  by 
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lemma  22.  From  the  definitions  of  F(T,Xq)  and  X(t,Xo)  we  have  formally, 

Fx  [T,X0]  -  X[T,X0],  (127) 

o 

which,  by  assumptions  (123i)  and  (124)  and  lemma  24,  exists,  is  continuous  and 

*  ★  ~ 

bounded  in  some  neighborhood  of  (T,Xq(T)).  Furthermore,  whenever  Tj£T(XQ), 
from  (98), 

FtIT,Xq]  *  Ax(T,\q)+  Bu(T,Xq).  (128) 


Hence,  by  assumption  (123),  F^fTjX^]  is  continuous  and  bounded  in  some  neighbor- 

^  ic  * 

hood  of  (T,X  (T)).  Therefore,  (102)  is  satisfied.  Since  F,  (T,X  )  and 
O  A  O 

Ft(T,Xo)  exist  and  are  continuous  in  a  neighborhood  of  (T,Xq)  ,F(T,Xo)  is  differenti¬ 
able  in  that  neighborhood,  satisfying  (103) .  Finally,  assumption  (125)  and  (127) 

imply  [F^  [T,X*(T)]}~^  exists, 
o 

Theorem  4  then  implies  the  existence  of  a  function  G(T)  satisfying 
(105)-(108) .  It  remains  to  be  shown  that  Xq(t)  =* G(T) ,  i.e.  if  G(T)  is  used 
as  an  initial  condition,  the  resulting  control,  u[t,G(T)]  is  in  fact  optimal 
for  T€^(T).  This  is  in  fact  the  case  since  F[T,G(T)]  =0,  by  lemma  21.  (126) 

is  (108)  using  (127)  and  (128).  Q.E.D. 

*  dJ* 

Now,  returning  to  the  properties  of  J  (T)  and  in  (93),  consider 

i..  >Tl\  ^ 

the  integrand  k'  — ^  *-  .  If  Xo(T)  generates  an  optimal  control  for  each  T 
through  equations  (94)  to  (96),  then 

u*(t,T)  »  u^[t,X*(T)]  Vt  i  J[X*(T)].  (129) 


Formally,  we  have 

^(t.T) 

dT  T*T 


“iIt,X*(T  +  A)J  -uJt.X^T)] 

A 


lim 

A-0 


(130) 
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which,  by  the  piecewise  constant  nature  of  u^(t,Xo),  is  zero  for  all 
tjt  T[X*(T)]  if  X*(T)  is  continuous  in  a  neighborhood  of  t.  Consider  (130) 
at  a  switch  point  ty  6  J[Xq(T)]  n.(0,T)  and  suppose  i  ^ Y I tv> (T) ] .  Considering 
tv*t^(\o),  as  a  function  of  the  costate  initial  condition,  under  assumption 
(119),  the  sensitivity  of  t*(XQ)  with  respect  to  Xq  is  given  by  (120).  Hence, 
provided 

~  it 

(131) 


b'A'\[tv,\*(T)]  /  0, 


the  variation  of  the  switch  time  of  the  ith  component  of  the  control  with 
respect  to  T  at  1*1,  may  be  written  as 


at1 


dr 


_  <(T)  ■  \ 
Xo-X*(T)*  dT  T-T 


T  / 


(132) 


dtJ 


with  -gr—  given  by  (120)  and,  provided  the  assumptions  of  theorem  5  hold, 
<(T),° 

0  -|  given  by  (126) 


dT  x=T 

The  evaluation  of  the  limit  in  (141)  leads  to  delta-functions  at 
each  tv €T[X*(T)] ,  i.e.,  in  a  neighborhood  of  t^  €X[X*(T)]  H  (0,T),  ^(t^),  we 
have 


aMt.i)  ,  atv 

“If -  '  *  "  6(t-tv)  •  3T*  *  for 

WA  T*T  x  t*T  v 


(133) 


hr  * 

where  is  given  by  (132).  The  full  evaluation  of  — ^  may  be  expressed 

in  terms  of  6-functions  and  then  substituted  into  (93)  to  evaluate 


Applying  Leibniz'  rule  in  such  a  case  with  generalized  functions  is,  in  fact, 
valid  and  the  following  result  could  be  proved  in  that  manner  (see  p.  77,  [17]) 
However,  the  following  theorem  will  be  proved  directly  by  considering  the  limit 
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of  Che  enCire  integral. 

Theorem  6.  Given  a  normal  opCimal  control  problem  with  state  equation  (85), 

* 

control  constraint  (86),  boundary  conditions  (68)  and  cost,  (87),  let  J  (T) 
be  the  minimal  value  for  the  cost  for  time  interval  [0,T].  If  for  T>0,  3  an 
initial  costate  XQ  such  that  the  control  u(t,\Q)  generated  by  (94)-(96)  is 
optimal  for  T»T  and  if 


T/ET(Xq), 


(134) 


and  for  all  tv  £T(£o)  f]  [0,T]  with  i6Y(t^,to), 


bjA*  X(tv,XQ)  +  0 


(135) 


and 


X(T,X  )  is  non-singular. 


(136) 


then,  J  (T)  is  continuous  and  differentiable  in  a  neighborhood  of  T  with 
dj* 

•^-1  „  given  by 


T-T 


1  -  -k-ad.X  )  +  [[X(T,X  )]'1[a£+bS(T,X ,  )]}' 

T"T  fm  Ni<T’\>>l 


x  x  L  H 
x  U=1  j-1 


where  X(t,XQ)  Is  defined  in  (109), 


.'“l  1 

-A't} 

.b!  A'e  JX  . 

i  o 

w  = 


-1  if  fi(0,Xo)i0 


1  if  fi(0,Xo)<0 


(137) 


(138) 


Ni(T’V  of  ^TCXo}  n  [0,TJ3  Y(tv,Xo)  =  i 


(139) 


and 
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(140) 


.  N. (T,X  )  . 

ItjJjii  0  -  U^TO^nco.u  lY(tv,io)-i] 

with  OSt|<...<  t^  ^  ^  ^  s  T  for  i-l,...,m. 

Proof.  If  the  derivative  exists, 

-  lim  J  (^A)  -T-.J  Sll  m  iim  i{  j*+Ak'u*(t ,T+A)dt  -  A'u*(t,T)dt} 
3T  T-T  A-MD  A  A-*0  A  0  0 

-  Iim  -k  f*+Ak,u[t,X*(T+A)]dt  -  /\'u[t,X*(T)  ]dt } 


A-K) 


A  o 


^  A  A  "ff 

where  X  (T)  ■  X  ,  and  X  (T)  is  the  continuous,  differentiable  function  in  a 
o  o  o 

neighborhood  of  T  which  generates  through  (94)  to  (96)  the  unique  optimal 

it 

control,  u  (t,T)  *  u[t,X  (T)]  for  the  time  T.  Xq(T)  is  guaranteed  to  exist  by 
Theorem  5.  Now,  for  A  >  0, 

J  .(T+A),  - . J_(T)  ,  1  *(T+A)  ]dt  +  i  f  V  (u[t,X*(T+A)  ]  -  u[  t ,  X*(f )  ]  }dt . 

(141) 


Since  T  £T(X  ),  the  first  term  clearly  gives 


T+A 


*  - 


lim  f  k'u[t, X  (T+A) ]dt  -  k'u[T,X  ] 
A-0+  T  0  0 


(142) 


•ft  A  A  A  A  A 

because  uft,X  (T+A)]  i  u[T,X  (T) ]  in  a  neighborhood  of  T  by  the  continuity  of 
o  o 

* 

the  switch  functions  (95)  and  the  continuity  of  X  (T). 

Now,  the  continuity  of  the  switch  times  WRT  T  further  implies  that 
given  £  >  0,  3  A6  >  0  3  T  0  <  A  <  AE, 

N(T , X  )  -  N[T,X*(T+A) ] ,  (143) 

o  o 

and 

tv[X*(T+A)]  €  (tv(Xo)  -e,  tv(XQ)  +  e)  (144) 

For  a  normal  problem,  the  number  of  switch  times  in  a  finite  interval  is  finite 
(this  is  proved  in  a  later  lemma)  so  that  e  may  be  chosen  small  enough  that 
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<tv1<Xo)  ‘  e*  \(x>  +  £)  °  (t,2(V  -  6*  W  +e) 

V  v,  ,  v,  €{1 . N(T,X^)  },  jt  \>2 • 


(145) 


1*  2  -  o' 

Therefore,  for  0  <  A  <  A  ,  the  second  term  in  (141)  is, 

it  *  *  N(T,X  )  "t  ^  A  * 

j  f\'{u[t, -o(f+A)]  -  u[t,Xo(T)}}dt  -  j  vIL  °  ItV  k’{u[t,Xo(T+M]  -  u[t,Xo(T)]}dt, 

(146) 

where 

t  ■  max{t  -  G,  0}  and  t  ■  min{t  -  e,  T}. 


—v  v  '  v  v 

Now,  suppose  u.(t,Xo)  switches  from  0  to  u^  at  tv(XQ).  Then 
.t 
t 


J  ^{^[t.X^T  +  A)]  -uirt,\*(T)]}dt--kiui[tv[\*(T+A)]  -  t  tfo)] 


and  similarly  if  u^(t,\o)  switches  from  u^  to  0, 


.  w  ^  ^  ^ 

Jt\^iIt^0(T  +  4)]  -ui[t,Xo(T)]}dt=kui[tv[Xo(T  +  A)]  -  t  (X  )], 

i  * 


Since  uj_ ( c » XQ )  switches  between  0  and  u^  according  to  (96),  if  we  consider  only 
the  tv€T(\Q)  where  the  ith  component  of  the  control  switches,  then,  using 
definitions  (138)  to  (140)  we  have 

MT.O.t1 


1  j-l°  J  i  kit“iIt»Xo<i  +  A)]  -^[t^oWJJdt 

Ni(i^o) 

"  'WVi  j-!  Uj[k*(f  +  A)J  -tJ[X*(T)]}.  (147) 


-j 


Dividing  (147)  by  4,  taking  the  limit  as  A“*0+,  and  summing  over  all  i*l,...,m 
gives , 

T  m  Ni(T,\o)  dti 

liji  i‘ol.'t;it,^(i+4)l  -Ile^CTHidt.-^  51d0)»1fcl(^l  .^1  > 

N  O  O 


1 


From  (120),  (126)  and  the  fact  that  the  solution,  A(t,XQ),  to  (111)  is  given 
by  e  A  C  and  X(t,XQ)  •  e  ^  CXo,  from  (94), 

oX*(T)  \  .  .  _!  .... 

-x  •  ■‘Ux(I,x°)l 


“A  t,  | 
e  b.J. 


b^A'e 


Combining  the  last  two  equations  gives  the  second  term  of  (137). 

For  A~*0  the  above  analysis  applies  except  that  for  4<0  we  write 


J*(T  +  A)  -  J*(T) 
A 


T  J  k«[G[t,X*(T  +  4)]  -u!t,X*(T)]}dt 


7  j\  k*u[t,X* (T  +  A)]dt. 

T+  4 


(148) 


The  last  term  gives  the  same  limit  as  in  (142)  and  clearly  the  first  term 

approaches  the  second  term  of  (137)  using  the  same  argument  applied  to  (146). 

★ 

Hence,  the  left  and  right  derivatives  of  J  (T)  exist  and  are  equal,  proving 
the  theorem.  Q.E.D. 

The  coordinator  may  use  the  result  of  theorem  6  to  compute  the 
variation  of  the  off-line  costs  WRT  variations  in  the  time  intervals  dictated 
by  v(t).  Since  this  computation  depends  upon  knowledge  of  the  optimal  control 
for  a  given  time  interval,  methods  for  solving  the  optimal  control  problem 
must  be  available.  One  such  method  is  that  of  Stefenale  [16]  which  employs  a 
gradient  search  or  Newton's  method  to  find  the  optimal  initial  costate. 


5.  APPLICATION  10  THE  UNIT  COMMITMENT  PROBLEM 
5,1,  Application  of  the  General  Results 

The  generic  problem  description  of  Section  1.2  contains  most  of  the 


salient  features  of  the  thermal  unit  commitment  problem  for  power  systems  as  it 
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is  normally  formulated.  The  model  developed  in  Section  1.2  has  the  added 
feature  of  providing  for  a  more  precise  modeling  of  the  off-line  control, 
as  discussed  in  Section  1.1.  The  primary  element  of  the  unit  commitment 
problem  not  included  in  this  formulation  is  the  stochastic  nature  of  the  pre¬ 
diction  of  demand  and  predicted  machine  availability.  Although  these  are 
important  considerations,  they  were  not  taken  into  account  in  this  initial 
inquiry  since  even  the  deterministic  problem  demands  further  research  and 
currently  such  stochastic  aspects  are  disregarded  or  included  only  in  a  very 
heuristic  manner  in  practical  optimization  schemes. 

By  separating  the  problem  into  the  two  natural  divisions  of  a 
steady-state  optimization  (Section  3)  and  several  dynamic  optimal  control  pro¬ 
blems  (Section  4),  the  relation  of  the  general  formulation  to  the  usual  approach 
to  unit  commitment  is  evident.  The  pointwise  optimal  coordination  of  the  on¬ 
line  subsystems  corresponds  to  what  is  known  as  economic  dispatch  for  the  power 
system  application.  Economic  dispatch  is  well  developed  theoretically  [18]  and 
is  almost  universally  used  in  some  form  throughout  the  electric  utility  industry 
[19].  On  the  other  hand,  the  formulation  and  modeling  of  the  shut-down  and 
start-up  of  thermal  generating  units  as  an  optimal  control  problem  is  done  in  a 
very  rough  manner  in  the  literature  and  requires  considerable  research  [20]. 

The  computation  of  the  off-line  costs  is  currently  done  using  an  empirical 
determination  of  the  parameters  for  a  simple  start-up  cost  vs.  down-time  formula 
representing  an  exponentia  1  cooling  rate  or  some  computationally  advantageous 
approximation  there  of  [3,21].  This  modeling  problem  is  discussed  further  in 
Section  5.2. 

Once  a  reasonable  model  is  available,  the  results  of  the  previous 
sections  may  be  employed  to  enhance  and  extend  current  unit  commitment 
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optimization  schemes.  One  possible  application  of  the  results  of  Section  3 
would  bfe  to  use  the  properties  of  the  optimal  cost  functions  given  there  to 
enhance  the  dynamic  and  integer  programming  algorithms  which  are  currently 
used  to  find  sub-optimal  conmitment  policies.  This  is  mentioned  briefly  in 
Section  5.3  as  a  area  for  further  research. 

Another  application  of  the  results,  particularly  theorems  1  and  6, 
would  be  to  compute  the  gradient  of  the  cost  for  a  feasible  commitment  policy 
with  respect  to  the  switch  times  of  v(t).  Since  existing  algorithms  compute 
a  suboptimal  v(t),  that  could  be  used  an  an  initial  feasible  conanitment  policy 
for  which  the  gradient  of  the  cost  with  respect  to  the  switch  times  could  be 
used  for  a  gradient  search  among  the  switch  times  for  a  local  minimum  around 
that  initial  v(t).  This  would  result  in  an  improved  cost  with  relatively 
little  added  computation.  Since  current  solution  techniques  discretize  the 
time  interval  into  hour  segments,  the  subsequent  gradient  search  would  "fine- 
tune"  the  result  by  allowing  for  switching  the  generating  units  on-line  and 
off-line  at  any  time.  It  is  noted  that  such  a  gradient  search  method  is 
essentially  what  is  used  in  [3],  but  for  a  much  simpler  model. 

The  formula  for  the  gradient  of  G^vCt)]  of  (15)  with  respect  to  the 
switch  times  of  v(t)  is  given  in  theorem  8  below.  Given  an  initial  feasible 
commitment  function,  v^t),  the  sets  of  units  which  are  to  be  committed 
between  switch  times  are  fixed  and  only  the  switch  times  are  varied.  If 
v^t)  has  K  switch  times  in  (0,T),  given  by  [ t° j  with 

0<tl<t2<”- <Ck<T,  (149) 

then  v  (t)  is  given  by 
■^o 

v,(t)-vk  for  tk-iSt<tk»  k“l . K+l  (150) 


V,  i i  trt  •  *, 
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Finally,  to  designate  the  switch  times  for  the  individual  subsystems, 
the  following  definitions  are  made.  For  k»l,...,K  define  the  sets  and 


h as 


^  -  U  I  *  1,  v*+1  -  0] 

4^  *  [i  I  v^  ■  0,  v£+1 *  1} 

and  for  the  jth  iteration  define  for  i*l,...,N  and  k«l,...,K, 

.  ■  maxi  I  l  ■  0, . . .  ,k-l,  i  c  4  ,j 
k,  i  x,  * 

^  *min{t^  1  l  »k+l,...,K+-l,  it^j 

<Vr 

With  these  definitions,  theorem  8  can  be  stated  giving  explicitly. 

K 

Theorem  8.  For  (^(jt/y)  defined  above,  the  gradient  with  respect  to  JtcR  is 
given  by 


(155) 


(156) 


dC-l 

*  t-tJ 


C[D(t|),v2J  -C[D(t{),v1]  + 

•  •  • 

•  •  • 

L5D<4>W  ■CtDtCK»^K^  +  Ail 


(157) 


where,  for  k**l,...,K, 


.  j  ^£i  I  £  ^Ji| 

^  '  1  fl\  M  T-tJ-xJ  ,  '  1  '*k  ^ i-tJ  ,-eJ 

k  k,i  k,i  k 


(158) 


and  is  given  for  the  ith  subsystem  by  (137)  of  theorem  6. 

Proof.  Straight  forward  application  of  theorems  1  and  6  to  the  definition 
(15)  of  the  total  cost. 
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5.2.  Modeling  of  Off-Line  Thermal  Units 

In  this  section  some  brief  preliminary  discussion  is  presented 
concerning  the  dynamic  modelling  of  the  thermal  generating  units  while  off¬ 
line.  Previous  work  on  determining  start-up  costs  usually  assumes  the  cost 
of  start-up  depends  linearly  upon  the  difference  between  the  boiler  operating 
temperature  during  generation  (on-line  steady-state  operation)  and  the 
temperature  to  which  the  boiler  has  cooled  since  it  has  been  taken  off-line. 

The  exponential  cooling  rate  assumed  was  mentioned  above.  Since  such  a  model 
does  not  include  any  dynamics  which  account  for  the  time  needed  to  reheat  to 
boiler  to  operating  temperature,  that  time  factor  is  usually  accounted  for  by 
assuming  a  fixed  time,  normally  an  hour,  for  start-up  regardless  of  the  state 
of  the  system  at  the  time  start-up  is  initiated. 

Since  virtually  no  information  concerning  dynamic  modelling  of  the 
start-up  process  is  available  in  the  literature,  a  simple  linear  model  with  a 
single  state  is  proposed  below  which  takes  into  account  the  simpler  cost 
approximations  discussed  above  but  also  includes  a  more  reasonable  and  flexible 
model  of  the  start-up  process.  Some  data  from  a  particular  industry  applica¬ 
tion  [21]  is  presented  to  show  the  range  of  values  for  the  cooling  time 
constant,  however,  data  for  the  start-up  time  constants  are  unavailable.  Hence, 
even  the  simple  model  suggested  below  requires  further  evaluation  with  respect 
to  current  practice. 

Consider  a  lumped  parameter  model  of  the  boiler  dynamics  with  a 
single  state,  T,  representing  the  average  temperature  of  the  water.  The 
temperature  has  been  previously  used  as  the  state  of  the  thermal  system  [12] 
since  in  a  gross  sense  it  represents  the  energy  content  of  the  system.  With 


A 
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no  heat  input,  the  cooling  rate  depends  upon  the  difference  between  the 
boiler  temperature  and  the  ambient  temperature.  Following  the  usual  assump¬ 
tion  of  an  exponential  cooling  rate,  if  TA  is  the  average  abient  temperature, 
the  differential  equation  satisfied  by  t  is 

T  =  -a(T  -  Ta) 

where  a  is  the  inverse  of  the  cooling  time  constant.  Reasonable  values  for 
a  range  from  4  to  12  hours  [21]. 

Now,  suppose  the  sole  control  available  is  the  fuel  rate  to  the 
burners  heating  the  boiler.  If  u  is  the  fuel  rate  in  units  of  volume/time, 
and  p  is  the  conversion  constant  of  heat  content  per  volume  of  fuel,  where  it 
is  assumed  the  heat  transfer  to  the  boiler  is  constant  over  the  operating 
temperatures,  then  the  single  state  model,  including  control  is 

t  -  -or(T  -  ta)  +  pu. 

It  is  assumed  the  fuel  rate  is  bounded  by 

OSuSu. 

Finally,  the  operating  temperature  is  denoted  by  T. 

Since  it  is  the  difference  between  the  boiler  and  ambient  tempera¬ 
tures  which  matters,  a  new  state,  x,  may  be  defined  as 

X  =  T  -  T  .  (159 

Furthermore,  let  x  =  T  -  t  .  If  the  cost  in  $/volume  for  the  fuel  is  given  by 
k,  then  the  total  cost  for  operating  the  generating  unit  over  a  time  interval 
[0,T]  is  given  by 
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J(u,T)  =J  kudt 


(160) 


Hence,  the  optimal  control  problem  for  an  off-line  unit  can  be  stated  as 
follows.  Given  the  off-line  interval  [0,T],  minimize  J(u,T)  of  (157)  subject 


to 


X  =  -OX  +  Pu 


(161) 


with  the  boundary  conditions 

x(0)  * x,  x(T)  =  x. 


(162) 


Clearly  the  results  of  Section  4  can  be  immediately  applied  to  this 

* 

model.  In  fact  the  optimal  control  u  (t,T)  is  given  by 

f 


u  (t,T)  =  ( 


0  s  t  <  t 


u  t  stsl 

S 


where 


(163) 


(164) 


This  control  produces  the  optimal  trajectory,  x  (t,T),  given  by 

/ 


•k 

x  (t,T) 


—  -at 

x  e 


0st£t 


xe'at+  -fd-e  S>)  ts<tST 


(165) 


Since  the  state  is  a  scalar,  we  have  by  lemma  16  that  the  optimal  cost  as  a 

k 

function  of  T,  J  (T) ,  is  monotone  increasing  and  continuous.  From  (159)  and 

d«J* 

(163)  it  is  easily  shown  that  is  given  by 


bJ_ 

St 


ku[l  +  eaT(  H-  l)]'1 
1  'ax 


(166) 
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which  is  always  positive. 

In  words,  the  optimal  policy  is  to  let  the  boiler  cool  until  the 
time  ts  when  there  is  just  a  sufficient  amount  of  remaining  time  before  the 
unit  is  needed  to  reheat  the  boiler  to  the  operating  temperature  using  the 
maximum  fuel  rate.  Of  course,  this  is  roughly  what  is  done  in  practice.  It 
may  be  the  case,  however,  that  for  a  more  thorough  model  of  the  boiler  dynamics 
including  time  constants  for  heating  boiler  walls,  multiple  burners,  circula¬ 
tion  rates,  and  a  more  detailed  lumped  parameter  model  of  the  governing 
partial  differential  equations ,  the  process  will  not  only  be  better  represented 
but  there  may  be  control  policies  suggested  by  such  an  analysis  which  would 
improve  upon  current  practice.  A  detailed  modelling  of  the  boiler  and  turbine 
dynamics  for  use  in  modeling  the  generating  unit  while  on-line  is  given  in 
[22]  and  [23].  This  may  serve  as  a  general  guide  for  some  of  the  important 
points  which  must  be  considered  for  a  modeling  of  the  start-up  process,  but 
there  are  certain  features  which  are  irrelavent  to  modeling  the  system  while 
off-line. 

It  should  be  noted  that  one  other  aspect  of  the  cost  of  taking  units 
off  line  and  returning  them  to  service  which  is  not  included  in  the  above 
discussion  is  the  fixed  costs  which  may  be  incurred  each  time  such  a  switch 
occurs.  Such  costs  may  result  from  machine  wear  or  labor  expenses  but  these 
were  not  included  because  the  added  theoretical  difficulties  which  arise  in 
including  these  factors  are  far  to  great  compared  with  the  importance  and 
magnitude  of  such  costs.  In  particular,  to  include  fixed  costs  for  each  switch 
in  the  continuous  model  proposed  would  require  the  used  of  6  '  functions  in  the 
integrand  and  the  theory  for  optimization  with  such  costs  becomes  far  more 
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complex  than  that  presented  above  [24] .  Because  these  costs  are  disregarded 
it  is  noted  that  the  optimal  solution  is  never  to  "bank"  the  boiler,  i.e,  to 
maintain  the  boiler  at  its  operating  temperature  while  it  is  off-line.  In 
practice  this  is,  of  course,  often  done  and  may  be  the  cheapest  policy. 

5.3.  Directions  for  Further  Research 

The  following  list  summarizes  those  areas  which  were  discussed  in 
previous  sections  as  directions  for  further  investigation. 

1.  Using  the  results  of  Section  3  to  enhance  and  improve  the  guide¬ 
lines  employed  by  existing  algorithms  which  search  among  the  feasible  sub¬ 
system  combinations  for  an  optimal  solution.  Any  practical  algorithms  must 
exclude  the  majority  of  options  by  some  method,  such  as  branch  and  bound 
techniques  when  the  problem  is  formulated  as  an  integer  programming  problem. 

The  exclusion  leads  to  suboptimal  solutions  and  perhaps  the  knowledge  of  the 
properties  of  the  steady  state  cost  function  would  be  useful  in  determining 
the  appropriate  subset  of  units  which  should  be  considered  at  each  demand 
level. 

2.  Evaluation  of  the  value  of  incorporating  the  suggested  gradient 
search  method  into  existing  schemes  for  optimization.  Clearly,  with  respect 
to  the  unit  commitment  problem,  it  must  be  determined  whether  or  not  extending 
existing  methods  results  in  enough  improvement  to  justify  the  added  computation. 
Since  all  present  algorithms  lead  to  suboptimal  solutions  and  in  fact  may  lead 
to  poor  solutions  under  some  circumstances,  it  appears  that  such  an  extension 

is  worthy  of  consideration  since  it  can  be  incorporated  with  little  change  to 
existing  software. 
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3.  Include  the  probabilistic  and  stochastic  aspects  of  the 
problem  in  the  formulation.  This  would  involve  essentially  defining  &  new 
problem.  Very  basic  issues  must  be  resolved  concerning  appropriate  defini¬ 
tions  for  the  cost  function  under  stochastic  assumptions  as  well  as  defining 
what  is  meant  by  "meeting  the  demand"  or  "a  feasible  set  of  units".  The 
approach  taken  for  such  research  should  be  closely  related  to  the  application 
at  hand  since  even  simple  formulations  quickly  lead  to  complex  problems  when 
optimization  is  attempted  and  consequently  the  assumptions  made  in  the  model 
should  represent  a  real  problem  so  that  the  effort  is  not  meaningless.  Never¬ 
theless,  it  is  important  that  these  factors  are  investigated  since  it  is  the 
case  that  a  strictly  deterministic  model  ignores  a  fundamental  property  of  the 
power  system  application. 

4.  Include  fixed  costs  for  switching  units  off-line  and  on-line. 

As  was  discussed  at  the  end  of  Section  5.2,  it  is  questionable  that  these 
cost  factors  merit  the  added  theoretical  difficulties  incurred  by  including 
them,  however,  research  in  that  direction  could  be  intended  to  find  ways  of 
including  these  factors  in  perhaps  a  more  subtle  but  computationally  attractive 
manner.  In  practice,  since  discrete  time  models  are  considered,  the  problems 
encountered  with  the  continuous  time  model  discussed  here  do  not  arise. 

5.  The  practical  problem  of  modeling  thermal  unit  shut-down  and 
start-up.  This  problem  which  was  discussed  at  length  in  Section  5.2  is 
presently  in  its  infancy  and  demands  an  investigation  of  industry  practice. 

It  is  a  very  practical  problem  which  cannot  be  addressed  without  close  consulta¬ 
tion  with  power  system  operators. 

In  summary,  this  report  presents  preliminary  results  and  considera¬ 
tions  concerning  a  very  complex  and  important  problem.  The  list  given  above 


-♦  -■  -  -  -  -----  -  -  —  — - -  —  -  - 
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is  merely  a  sampling  of  the  many  appropriate  directions  for  further  research 
addressing  the  general  system  coordination  problem  and  the  application  of  unit 
commitment. 
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